-- Logs begin at Thu 2022-08-18 20:51:53 UTC, end at Thu 2022-08-18 22:44:02 UTC. -- Aug 18 21:40:46.706408 dv-cinder-853678-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 18 21:40:46.718210 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 18 21:40:46 2022] *** Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: nodename: dv-cinder-853678-3-iscsi Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: machine: x86_64 Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: clock source: unix Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: pcre jit disabled Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: detected number of CPU cores: 4 Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: current working directory: / Aug 18 21:40:46.720289 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: detected binary path: /usr/bin/uwsgi-core Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: your processes number limit is 47680 Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: your memory page size is 4096 bytes Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: detected max file descriptor number: 2048 Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: lock engine: pthread robust mutexes Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: thunder lock: enabled Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 18 21:40:46.720816 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 18 21:40:46.760506 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: Python main interpreter initialized at 0x558ab3d969c0 Aug 18 21:40:46.760506 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: python threads support enabled Aug 18 21:40:46.760506 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: your server socket listen backlog is limited to 100 connections Aug 18 21:40:46.760506 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: your mercy for graceful operations on workers is 90 seconds Aug 18 21:40:46.760881 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: mapped 403077 bytes (393 KB) for 2 cores Aug 18 21:40:46.760963 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: *** Operational MODE: preforking *** Aug 18 21:40:46.761192 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: *** uWSGI is running in multiple interpreter mode *** Aug 18 21:40:46.761192 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: spawned uWSGI master process (pid: 81446) Aug 18 21:40:46.761400 dv-cinder-853678-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 18 21:40:46.762637 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: spawned uWSGI worker 1 (pid: 81458, cores: 1) Aug 18 21:40:46.762637 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: spawned uWSGI worker 2 (pid: 81459, cores: 1) Aug 18 21:40:46.762637 dv-cinder-853678-3-iscsi devstack@keystone.service[81446]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 18 21:40:51.049946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 21:40:51.049946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Aug 18 21:40:51.050845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 21:40:51.050845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Aug 18 21:40:51.051211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 21:40:51.051211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Aug 18 21:40:51.051806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 21:40:51.051806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Aug 18 21:40:51.100617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 21:40:51.100802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 21:40:51.100987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] command line args: [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 21:40:51.101153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 21:40:51.101243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 21:40:51.101353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 21:40:51.101439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ================================================================================ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 21:40:51.101551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] command line args: [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 21:40:51.101674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.101852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 21:40:51.101933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_dir = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ================================================================================ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 21:40:51.102128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_source = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] debug = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_dir = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.102953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_source = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] debug = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.103892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_config_append = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_dir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_config_append = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.104978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_dir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.105991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.106885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.107937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] publish_errors = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.108949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] publish_errors = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.109930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] standard_threads = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] standard_threads = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_journal = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_json = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.110978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_stderr = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_syslog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_journal = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_json = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_stderr = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.111785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_syslog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.111921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 21:40:51.112181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.112984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.113966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.114918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.115883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.116881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.117863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.118980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.119932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.120949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.121999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.122990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.123929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.124934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.125911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.126870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.127974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.128925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.129980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.130931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.131888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.132987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.133942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.134946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.135910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.external = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.136850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.137036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.137114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.137801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.137918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.password = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.external = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.token = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.138908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.password = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.token = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.139949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.140891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.141888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.142883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.143871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.144987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.145972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.146939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.147970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.148910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149539 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.149887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.150835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151836 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.151999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.152986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.153670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.153813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.153899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.153972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.154949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.155968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.156926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.157918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.158928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.159980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.160940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.161899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.162833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.163971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.164925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.165952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.166900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.167848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.168970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.169946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.170982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.171932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.172892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.173868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.174847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.175930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.176935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.177909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.178889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.179976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.180940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.181920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.182950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.183914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.184919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.185895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.186996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.187953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 21:40:51.188657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.188898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.189146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.189381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.189643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.189887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 21:40:51.190060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 21:40:51.218640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.218868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.219094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.219362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.219601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.219833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.227540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.228493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.255305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.255529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.256416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.256654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.309713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.310102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.358909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.359138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.359950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.360189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.517906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.518047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.518129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.518260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.518341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.541820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.542973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.543054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.543187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.543267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.543406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.554520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.554703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.567927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.568981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.569332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.569418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.569492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.569566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 21:40:51.586823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.587031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 21:40:51.587118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.587245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 21:40:51.587375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.587575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 21:40:51.587675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.587784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 21:40:51.587866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.588933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.589121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.598972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.599068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.599310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.605821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.606304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.609188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.609478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.609793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.609955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.610086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.610380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.610461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.610650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.610804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.610928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.611999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.612159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.612401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.612814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.613367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.613812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.614219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.635008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.636465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.636752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.642152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.642440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.642727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.649031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.649324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.650766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.653388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.653711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.653858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.654013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.654300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.654592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.654877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.655160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.655437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.655714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.659144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.659569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.660634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.669007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.669299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.673066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.673351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.675555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.675974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.676378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.676794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.677215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.677355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.677669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.678010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.678275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.680435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.680739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.685434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.688594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.691875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.692198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.692489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.692777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.693059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.693354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.699321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.699614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.704017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.707208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.707502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.709891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.710191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.710462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.710730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.711627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.714756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.717587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.717990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.718288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.718569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.718850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.719125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.719250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.719389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.719522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.722042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.722332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.722644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.722899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.723192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.734365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.734657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.734946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.735314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.735952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.736077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.742549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.744072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.744348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.747508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.747794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.748077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.748375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.748655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.751811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.754846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.757744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.759183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.759830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.760114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.760393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.760658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.760785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.760939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.761072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.761216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.761338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.761464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.768264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.769743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.772672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.772968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.773265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.773555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.777414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.780473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.781751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.782049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.782335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.783322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.784754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.786318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.786612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.786910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.789255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.789547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.789858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.790143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.790422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.790708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.790985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.791257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.791539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.791824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.792112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.792378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.792656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.792929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.793259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.795280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.798249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.798549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.798838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.799119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.806930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.807223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.807506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.816290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 21:40:51.816616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.817114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.817627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.818166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.818567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.819028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.819438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.819735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.820023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.820317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.820587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.820872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.821138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 21:40:51.827730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558ab3d969c0 pid: 81458 (default app) Aug 18 21:40:51.831226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af48d-b90a-407b-aca9-dad9d261d076 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:40:51.831519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af48d-b90a-407b-aca9-dad9d261d076 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:40:51.831797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af48d-b90a-407b-aca9-dad9d261d076 None None] PATH_INFO: `/v3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:40:51.835997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1/1] 10.222.0.82 () {50 vars in 791 bytes} [Thu Aug 18 21:40:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:40:51.854609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558ab3d969c0 pid: 81459 (default app) Aug 18 21:40:58.391647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:40:58.392319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:40:58.392319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:40:58.393584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 2/2] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:40:58.401167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:40:58.401541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:40:58.402004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:40:58.472746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: INFO dbcounter [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] Registered counter for database keystone Aug 18 21:40:58.492135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG oslo_db.sqlalchemy.engines [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81459) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 21:40:58.498352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writer thread running {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 21:40:58.678855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81459) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 21:40:58.679206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81459) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 21:40:58.734496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:40:58.736114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:40:58.864523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1/3] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:40:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 21:40:59.330608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:40:59.399563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: INFO dbcounter [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] Registered counter for database keystone Aug 18 21:40:59.421893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG oslo_db.sqlalchemy.engines [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81458) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 21:40:59.428063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writer thread running {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 21:40:59.548061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a72f-e93a-49f3-aade-ba1fbca8c935 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:40:59.552651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:40:59.553274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:40:59.553564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:40:59.553870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:40:59.562495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:40:59.614143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING keystone.common.rbac_enforcer.enforcer [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 21:40:59.614424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e100ff-68fe-41fb-9116-dd0c94f7b0e0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:40:59.624755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 3/4] 10.222.0.82 () {58 vars in 1324 bytes} [Thu Aug 18 21:40:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 21:40:59.630704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:40:59.645106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c03221-ea1a-4763-ab6d-778f7b972a2f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:40:59.649840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:40:59.650951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:40:59.651241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:40:59.651518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] PATH_INFO: `/v3/projects/940373c3ec634186892a510e45b0afa3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:40:59.653892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] RBAC: Authorizing `identity:get_project(project_id=940373c3ec634186892a510e45b0afa3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:40:59.706918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 21:40:59.707263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b65f1f-da75-44ac-908b-0c692f50f266 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:40:59.711910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 2/5] 10.222.0.82 () {56 vars in 1207 bytes} [Thu Aug 18 21:40:59 2022] GET /identity/v3/projects/940373c3ec634186892a510e45b0afa3 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:01.590050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243932a-2219-4837-9441-1d16c82f26b7 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:01.590580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243932a-2219-4837-9441-1d16c82f26b7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:01.590666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243932a-2219-4837-9441-1d16c82f26b7 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:01.594758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 4/6] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:01.600621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:01.601056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:01.601167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:01.621205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:01.623414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:01.631150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 3/7] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 21:41:01.892940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243932a-2219-4837-9441-1d16c82f26b7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:01.905962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243932a-2219-4837-9441-1d16c82f26b7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:01.910342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:01.910937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:01.911217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:01.911488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:01.919661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:01.921422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c383f49-8e48-41c1-b8ad-4a9ab22aa2fb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:01.930365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 5/8] 10.222.0.82 () {58 vars in 1324 bytes} [Thu Aug 18 21:41:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:41:01.935915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:01.944894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ec12-1978-4c95-9fc9-e092b88f8912 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:01.949170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:01.949918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:01.950240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:01.950511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:01.951803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:01.953679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d39b0f-838c-4e01-b582-dc2b0626f951 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:01.955194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 4/9] 10.222.0.82 () {56 vars in 1198 bytes} [Thu Aug 18 21:41:01 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:03.914402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2678-ab92-4477-af7b-c1cff08eada8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:03.914958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2678-ab92-4477-af7b-c1cff08eada8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:03.914958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2678-ab92-4477-af7b-c1cff08eada8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:03.916281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 6/10] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:03.921915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:03.922322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:03.922558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:03.924508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264652-bdf7-42dc-818a-62fd07a1683d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:03.924792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264652-bdf7-42dc-818a-62fd07a1683d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:03.925073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264652-bdf7-42dc-818a-62fd07a1683d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:03.926367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 7/11] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:03.931398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:03.931684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:03.931961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:03.941485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:03.942725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:03.975077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81458) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 21:41:03.975432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81458) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 21:41:04.021451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:04.023089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 5/12] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:04.023213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:04.120201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 8/13] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:04.292633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.317139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.388103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.398187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67f1f5-062e-4213-9b9a-c660a04ed3fd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.431779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.432421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.432705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.432980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.444584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:04.453955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:04.454998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:04.455837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 6/14] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:04.461826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.495158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48864fd6-23ef-4bcd-89fd-bdf0ec0f919d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.498076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.534838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.535601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.535881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.536151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.544899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.564078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60397ee1-5b57-447b-961f-37de1c45158f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.597439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.598092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.598456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.598636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.599314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:04.599597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:04.600980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:04.605494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:04.607535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:04.613124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 7/15] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:04.618817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.705634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 9/16] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 21:41:04.711105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.714677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.721787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a579a-d72d-414d-8902-3358f7d2ca84 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.757916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.758544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.758823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.759098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.760251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:04.761745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:04.763494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 8/17] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:04.768826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.782899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.812238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c7b5-0d2c-401b-8a1c-083ea3e3a0ef None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.844816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.845687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.845970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.846254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.865124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:04.866610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:04.868562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da426ba-bf3e-495f-81ff-18c7f97a16a6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:41:04.884778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:04.885730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:41:04.886549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:41:04.886549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:41:04.886549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:04.886549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 10/18] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:04.891680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.902449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:04.903597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:04.903872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:04.904138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:04.905252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:04.906722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:04.908247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 9/19] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:04.914145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:04.947572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:04.991669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205dea7-8181-4938-b1f6-14635b20f03e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:04.995010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:05.015064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb427-ce65-4fe3-8fb4-5dfece5a397f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:05.024322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:05.024940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:05.025232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:05.025510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:05.026187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:05.026494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:05.028645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054a5e-4bfd-40f6-9d71-b786aa037775 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:05.047414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 11/20] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:05.049730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:05.050664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:05.050955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:05.051228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:05.051969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.domain.id=default, user.id=29b6162ed5d14f26ae87d2e2d8c81d1b) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:05.052278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:05.053795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560746-471c-4d4c-9ad6-c7a4d5100150 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:05.061443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 10/21] 10.222.0.82 () {56 vars in 1322 bytes} [Thu Aug 18 21:41:04 2022] GET /identity/v3/role_assignments?user.id=29b6162ed5d14f26ae87d2e2d8c81d1b&scope.domain.id=default&role.id=a8f69336710f474db0973fac38e02b80 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:07.918746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae874b-92aa-4a66-b8fb-4282126b33c4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:07.919500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae874b-92aa-4a66-b8fb-4282126b33c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:07.919500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae874b-92aa-4a66-b8fb-4282126b33c4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:07.920941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 12/22] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:07.926087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351536-b652-4038-96cc-f93b42fcd1f5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:07.926769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351536-b652-4038-96cc-f93b42fcd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:07.926769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351536-b652-4038-96cc-f93b42fcd1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:07.951086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-30351536-b652-4038-96cc-f93b42fcd1f5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:07.952632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-30351536-b652-4038-96cc-f93b42fcd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:07.961293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb8c65-1045-48b5-a333-7c20eaace38e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:07.961833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb8c65-1045-48b5-a333-7c20eaace38e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:07.962620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb8c65-1045-48b5-a333-7c20eaace38e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:07.965290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 13/23] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:07.973559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5bd5-3d33-4b9a-868f-edbbd5cd44e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:07.974427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5bd5-3d33-4b9a-868f-edbbd5cd44e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:07.975186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5bd5-3d33-4b9a-868f-edbbd5cd44e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.034980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-67ed5bd5-3d33-4b9a-868f-edbbd5cd44e9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:08.037699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-67ed5bd5-3d33-4b9a-868f-edbbd5cd44e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:08.066231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 11/24] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:08.084240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431a24-6a9e-47c3-94c1-13814df6ef29 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.084710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431a24-6a9e-47c3-94c1-13814df6ef29 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.085181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431a24-6a9e-47c3-94c1-13814df6ef29 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.087851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 12/25] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:08.099913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.100389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.100876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.167227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:08.169746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:08.214629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 14/26] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 18 21:41:08.225705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7389-94a7-4c47-941a-0c02d77d6d46 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.226531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7389-94a7-4c47-941a-0c02d77d6d46 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.227288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a7389-94a7-4c47-941a-0c02d77d6d46 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.230688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 15/27] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:08 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:08.237440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.238681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.239394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.287829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:08.290253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:08.366943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 13/28] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:08.402493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:08.405302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:08.410144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:08.480397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 16/29] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:08.602192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293402b-d9f0-483a-8c9c-07a1d8f54a54 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:08.655668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:08.656321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.656603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.656878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.669255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:08.670766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:41:08.687989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:08.689297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:41:08.690655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:41:08.690655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:41:08.690655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:08.697624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 14/30] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:08.697897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:08.811717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:08.875622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54e47b-7857-4062-a070-091cc8a2dcb9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:08.898184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:08.954883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:08.955720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:08.955852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:08.956316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:08.956710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:08.957026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:08.958452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:08.959870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2578-2eb9-49a9-9c7e-897ff21f20d5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:08.992977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 17/31] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:08.993878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:09.006544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.007167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.007446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.007723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.008398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:09.009814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:09.086197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:09.167747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 15/32] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 21:41:08 2022] POST /identity/v3/roles => generated 225 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:09.169771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:09.236201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178f41-3bd7-46ab-9e3c-a16318d9f155 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:09.318249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.319460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.321185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.321889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.345769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:09.352608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:09.353518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:09.354315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 18/33] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:09.354935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:09.396535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8091af-3d25-400f-9d03-2c0c30b4c9a0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:09.416010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:09.439203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.439835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.440109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.440833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.441492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:09.443622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:09.489637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 16/34] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:41:09 2022] POST /identity/v3/projects => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:09.489637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:09.491391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:09.500675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afb2f2-a933-4f65-8206-4c970d359c9a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:09.555141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.555776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.556053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.556824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.557512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:09.560380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d90f6-aeb8-4fa3-ae1d-638e49218c62 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:09.623603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0c6f5-fb37-4d9b-992f-104d93cb178e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:09.665038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.665685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.665973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.666252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.666928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:09.667240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:09.668672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028b5cf-0bbd-4337-85c8-afc3aca70ec0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:09.679113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 17/35] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:09.680423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5110-98ca-422c-a857-3e78661df3b0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.680716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5110-98ca-422c-a857-3e78661df3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.680995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5110-98ca-422c-a857-3e78661df3b0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.682389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 18/36] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:09.683255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568345d-9ef6-4de8-a267-2e8fc29b85fb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.683599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568345d-9ef6-4de8-a267-2e8fc29b85fb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.684434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568345d-9ef6-4de8-a267-2e8fc29b85fb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.685825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 19/37] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:09.690038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568345d-9ef6-4de8-a267-2e8fc29b85fb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:09.695567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 19/38] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 21:41:09 2022] POST /identity/v3/roles => generated 219 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:09.697240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a5dc-31d2-4f5b-b036-a3c02fb00b93 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.697851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a5dc-31d2-4f5b-b036-a3c02fb00b93 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.698675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a5dc-31d2-4f5b-b036-a3c02fb00b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.723498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3973a5dc-31d2-4f5b-b036-a3c02fb00b93 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:09.725120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3973a5dc-31d2-4f5b-b036-a3c02fb00b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:09.732683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:09.821629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 20/39] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:09.821629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.821629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.821629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.840702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:09.842239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:09.925505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 21/40] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:09.937332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568345d-9ef6-4de8-a267-2e8fc29b85fb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:09.974024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:09.974665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:09.974954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:09.975234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:09.976373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:09.977908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:09.979502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 20/41] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:09 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:09.985040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:10.003584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:10.091032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbedf1-214d-4cb0-b745-4bcdf3107880 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:10.100031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:10.126081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:10.126724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:10.127005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:10.127285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:10.128998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:10.130622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:10.132382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 22/42] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:10.138300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:10.203064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:10.207240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeadefd-7556-4b9e-88ca-5e0eae166617 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:10.228117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:10.242666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:10.243287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:10.243570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:10.243847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:10.244517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:10.244603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee52d20-7e85-424c-8663-4a500da50a33 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:10.245959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:10.260946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 21/43] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 21:41:10 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:41:10.261672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:10.282916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:10.283776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:10.284730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:10.285023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] PATH_INFO: `/v3/domains/default/users/29b6162ed5d14f26ae87d2e2d8c81d1b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:10.287432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=29b6162ed5d14f26ae87d2e2d8c81d1b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:10.289580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:10.321627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 23/44] 10.222.0.82 () {58 vars in 1361 bytes} [Thu Aug 18 21:41:10 2022] PUT /identity/v3/domains/default/users/29b6162ed5d14f26ae87d2e2d8c81d1b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:41:10.384251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889fa9f-0c3d-4b0a-a73d-85fdf4945c14 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:10.427615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:10.428279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:10.428565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:10.428835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:10.430092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:10.431595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218abb36-0ab8-4a34-abfa-b0d452dd4d2d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:10.433490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 22/45] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:10.439648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:10.549000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cf7e-b2d9-4454-8986-554f693d60a9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:10.582666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:10.583282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:10.583564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:10.583838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:10.584478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:10.585946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a54950-aab1-4b22-a510-9028696d35c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:10.604908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:10.619439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 24/46] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:41:10 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:12.244549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b704e-35d7-4df4-91b0-51b492655d90 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.245070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b704e-35d7-4df4-91b0-51b492655d90 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.245169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b704e-35d7-4df4-91b0-51b492655d90 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.246488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 23/47] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:12.251721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a195-4692-46ed-895f-cb2d081390fc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.252114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a195-4692-46ed-895f-cb2d081390fc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.252420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a195-4692-46ed-895f-cb2d081390fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.270600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8d4a195-4692-46ed-895f-cb2d081390fc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:12.271780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8d4a195-4692-46ed-895f-cb2d081390fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:12.337974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 25/48] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:12.515777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d49f-d438-4d4a-a82b-333a1d216154 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.515980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d49f-d438-4d4a-a82b-333a1d216154 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.516256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d49f-d438-4d4a-a82b-333a1d216154 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.517589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 24/49] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:12.522763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.523059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.523339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.543841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:12.545037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:12.608450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129d49f-d438-4d4a-a82b-333a1d216154 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:12.610488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 26/50] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:12.636448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:12.713665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129d49f-d438-4d4a-a82b-333a1d216154 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:12.747489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:12.748120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.748408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.748684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.758640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:12.763966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:12.764786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:12.765861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 25/51] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:12.770779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:12.827007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:12.870624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3606fc9-dd0e-4cf0-bae0-a2da5f80a195 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:12.880966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:12.905714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:12.906334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:12.906617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:12.906896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:12.907568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:12.907874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:12.909294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:12.920806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 27/52] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:12.926305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:12.970200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:12.986689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9fe2f-db28-40d9-87d3-d08f6fb7d83e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:13.023618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:13.024267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:13.024553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:13.024829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:13.026007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:13.027807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:13.030207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 26/53] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:13.036048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:13.130368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d15e8-8e1c-40cb-8843-37a730491a3c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:13.138143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3884-0206-4995-a567-eaad4ac083cf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:13.147754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:13.166693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:13.167346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:13.167635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:13.167918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:13.169692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:13.171152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:13.172745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 28/54] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:12 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:13.175387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:13.176006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:13.176290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:13.176568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:13.177216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:13.178268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:13.178703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:13.206641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 27/55] 10.222.0.82 () {60 vars in 1140 bytes} [Thu Aug 18 21:41:13 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:41:13.291182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b507-9f9d-455e-855f-d90d6c2117c9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:13.326620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:13.327276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:13.327570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:13.327850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:13.329006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:13.330510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834c276-f931-48e6-ad7a-6039ac3a90a5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:13.332135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 29/56] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:13.338153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:13.347995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:13.445309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634fba3-5ad5-46b6-a62d-6c203e20f2e6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:13.479039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:13.479668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:13.479958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:13.480235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:13.481801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.domain.id=default, user.id=29b6162ed5d14f26ae87d2e2d8c81d1b) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:13.481982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:13.483415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961f3e-5259-46ed-965a-6ae2007f80cf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:13.489866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 28/57] 10.222.0.82 () {56 vars in 1322 bytes} [Thu Aug 18 21:41:13 2022] GET /identity/v3/role_assignments?user.id=29b6162ed5d14f26ae87d2e2d8c81d1b&scope.domain.id=default&role.id=a8f69336710f474db0973fac38e02b80 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:15.121440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eccf-5455-4a3e-a94a-8c3cd4fe7fc3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:15.121983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eccf-5455-4a3e-a94a-8c3cd4fe7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:15.122090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eccf-5455-4a3e-a94a-8c3cd4fe7fc3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:15.123366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 30/58] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:15.128533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:15.128896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:15.129098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:15.147685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:15.149224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:15.210347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:15.215665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 29/59] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:15.478436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8eccf-5455-4a3e-a94a-8c3cd4fe7fc3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:15.534900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:15.580160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8eccf-5455-4a3e-a94a-8c3cd4fe7fc3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:15.612223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:15.612852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:15.613133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:15.613408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:15.614554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:15.615988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cce130-888d-4edb-bbdd-307965930722 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:15.617544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 31/60] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:15.623737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:15.722036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f56e3-5393-410f-9cf3-2236295bdc4c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:15.755302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:15.755848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:15.756125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:15.756396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:15.757047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:15.758518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279c944-9ab3-48f8-8b08-f53d2ebdd95e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:15.784931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 30/61] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:15 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:41:18.751148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ed024-852d-4359-bd1f-cf4be4873df3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:18.752099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ed024-852d-4359-bd1f-cf4be4873df3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:18.752600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ed024-852d-4359-bd1f-cf4be4873df3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:18.754486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 32/62] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:18.759821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a42fd-9a8d-46c1-88a9-b5cfb3f55591 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:18.760692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a42fd-9a8d-46c1-88a9-b5cfb3f55591 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:18.761190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a42fd-9a8d-46c1-88a9-b5cfb3f55591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:18.782948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0e2a42fd-9a8d-46c1-88a9-b5cfb3f55591 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:18.784157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e2a42fd-9a8d-46c1-88a9-b5cfb3f55591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:18.797143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0daa-496f-430f-ad5c-4e09dd67ee69 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:18.797758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0daa-496f-430f-ad5c-4e09dd67ee69 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:18.798267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0daa-496f-430f-ad5c-4e09dd67ee69 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:18.800117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 33/63] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:18.806191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a4d9-afd4-4e62-a4da-50a938bd52e4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:18.806484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a4d9-afd4-4e62-a4da-50a938bd52e4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:18.806766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a4d9-afd4-4e62-a4da-50a938bd52e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:18.839343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fb62a4d9-afd4-4e62-a4da-50a938bd52e4 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:18.841179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fb62a4d9-afd4-4e62-a4da-50a938bd52e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:18.889207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:18.919920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 31/64] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:18.947328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 34/65] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:19.043456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6e2b-c816-4dd8-82b2-130c742e0001 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.044085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6e2b-c816-4dd8-82b2-130c742e0001 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.044573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6e2b-c816-4dd8-82b2-130c742e0001 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.046467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 32/66] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:19.051323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.051618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.051902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.078616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:19.079827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:19.137737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:19.167476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 35/67] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:19.224724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6e2b-c816-4dd8-82b2-130c742e0001 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.358098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6e2b-c816-4dd8-82b2-130c742e0001 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.384467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.413501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.414284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.414574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.414897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.433620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a046c44a-f9d4-4acd-bb37-130be066e093 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:19.441074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:19.442014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:19.442761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 33/68] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:19.443915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf331-8ec6-402b-868b-91489af40994 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.444369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf331-8ec6-402b-868b-91489af40994 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.444813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf331-8ec6-402b-868b-91489af40994 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.446281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 34/69] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:19.449145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf331-8ec6-402b-868b-91489af40994 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.509789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:19.517818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1134273-7790-4e57-86ba-35627a59d430 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.556864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.557803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.558238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.558515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.567917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf331-8ec6-402b-868b-91489af40994 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.568988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-80732a78-3f44-4d83-8862-fd529dd4091b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:19.573796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:19.575180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:19.575925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 36/70] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:19.577081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.577375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.577860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.596029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:19.597302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:19.608334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.609166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.609637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.610081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.610912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:19.611410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:19.613582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:19.626671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 35/71] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:19.627928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.634456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:19.670229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 37/72] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:19.671775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.739920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad4699-8cdc-4751-b691-14e355731faa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.750161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:19.777889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0810-11ae-4871-b059-329c35036971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.778543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0810-11ae-4871-b059-329c35036971 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.778806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0810-11ae-4871-b059-329c35036971 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.779088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a0810-11ae-4871-b059-329c35036971 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.789011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db8ad0-6905-46d2-a9e1-5289be145675 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.790036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a0810-11ae-4871-b059-329c35036971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-587a0810-11ae-4871-b059-329c35036971 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:19.796517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:19.797524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:19.798361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 36/73] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:19.798993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0810-11ae-4871-b059-329c35036971 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.830952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.831890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.832221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.832504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.833170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:19.833477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:19.834880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0a144-b28a-46ea-b600-987529d01739 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 38/74] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f660ae5-918e-418f-af11-629780f618e9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f660ae5-918e-418f-af11-629780f618e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f660ae5-918e-418f-af11-629780f618e9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 39/75] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:19.848016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f660ae5-918e-418f-af11-629780f618e9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.885536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:19.917064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a0810-11ae-4871-b059-329c35036971 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.952560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:19.953207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:19.953491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:19.953804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:19.954972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:19.956839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:19.958455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 37/76] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:19.959219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:19.969188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f660ae5-918e-418f-af11-629780f618e9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:19.995681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.003136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.003752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.004006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.004286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.004955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:20.005261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.006683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062b3e6-a9cd-44d5-b6a0-eb1992c0bf5b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.014547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 40/77] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.015938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.016226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.016501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.034040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:20.035225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:20.062093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0578ce-e257-49a7-acc3-bb185edf5fd2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.095898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.096538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.096815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.097091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.101197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 41/78] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:20.101989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.110691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.112246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.113801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 38/79] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:19 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.114565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.139639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.215579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.216297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc071ba-8b86-4f96-ab73-43bde3a1f894 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.224298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d0f2-295d-483d-a423-5e2c7811611e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.254905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050936-2850-413e-833c-fac5177b642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.255550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050936-2850-413e-833c-fac5177b642e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.255833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050936-2850-413e-833c-fac5177b642e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.256131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050936-2850-413e-833c-fac5177b642e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.260376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.260992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.261277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.261553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.266768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd050936-2850-413e-833c-fac5177b642e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-dd050936-2850-413e-833c-fac5177b642e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:20.272178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:20.273774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 42/80] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:20.274690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050936-2850-413e-833c-fac5177b642e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.338496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.340576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.367307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.382707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050936-2850-413e-833c-fac5177b642e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.416957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.417525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.417840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.418147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.419326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.420797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.422433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 43/81] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.423200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.438908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.446378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 39/82] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:20.447193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.525350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a4b62-f1ad-4d76-9359-c69bca4c588e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.546748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413be4-6238-40c2-b69b-2a347c424cc8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.559574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.560189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.560477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.560758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.580483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.581091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.581371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.581674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.582336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:20.582638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.584022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.584448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.592171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 40/83] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.592978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.630748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.632216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.650057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.696892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e299b-7176-4888-81e4-71f18eeb2f3a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.731676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.731958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 44/84] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:20.732316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.732597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.732810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.732902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.734020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.735468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.737939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 41/85] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.738658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.789428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.835444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9835-6a8a-421e-8f4d-37015a129869 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.837200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab10f6-ee98-42ab-a49e-291cecf9969a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.859090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:20.870651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.871352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.871638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.871918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.872448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:20.873049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.873155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:20.873328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:20.873620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:20.874537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.876162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 42/86] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:20.876928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:20.943927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:20.945408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:20.985355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1c221-4a6e-4028-98dc-f0a71c299cc5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:20.999934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.019467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.020103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.020384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.020717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.021872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.023338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.024900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 43/87] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.025673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.052978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 45/88] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:21.053881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.085158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.127374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7762e7-24a6-481d-9c16-69dd4f7c33a1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.153123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209fca-c46a-4908-b8e6-494faca4534c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.160019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.160657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.160940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.161218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.162438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.163916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.165563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 44/89] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.166341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.191885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.192507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.192795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.193074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.193750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.195190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.222360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 46/90] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:21 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:21.223221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.237995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.281135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936fed8-ad98-4865-b82c-57632c93652a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.316497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.326842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.327486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.327870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.328054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.328783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=29b6162ed5d14f26ae87d2e2d8c81d1b) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:21.329097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.330639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.336971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 45/91] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/role_assignments?user.id=29b6162ed5d14f26ae87d2e2d8c81d1b&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.338616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.354694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87c5b-687a-40eb-8d5a-ce7a65655ba9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.407106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.407759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.408044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.408326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.492503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dedc3-cd07-4c75-a0a6-22467a78078b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.515807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.517581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.530948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.537292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.538537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.538899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.539251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.540780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.543018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.545233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 46/92] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.546601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.632561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.640242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 47/93] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:21.641177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.685976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270f53-8ba4-4618-913e-2066ec0c15ad None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.731005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.731914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.732333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.732836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.733738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:21.734151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.735807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.742058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 47/94] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.743053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:21.785915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b658200-95eb-4b5c-a8c1-fac2cc3b9a4b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.802147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.841901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.842534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.850004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.850297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.851064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:21.851378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.852859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.868147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f8a5-f33d-40dc-94c1-01a6d028f7f2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:21.869728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 48/95] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.916532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:21.924185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:21.925063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:21.925359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:21.925667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] PATH_INFO: `/v3/projects/096974dcaa5a4bfe89dc919cde35dc64` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:21.926921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] RBAC: Authorizing `identity:get_project(project_id=096974dcaa5a4bfe89dc919cde35dc64)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:21.928398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86faef0-854d-4eab-b4fe-eedb5089bf96 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:21.930236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 48/96] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/projects/096974dcaa5a4bfe89dc919cde35dc64 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:21.937085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:22.115124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb700-699c-487b-a19f-4748b43d2414 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:22.154403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:22.155036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:22.155324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:22.155608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:22.156347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=096974dcaa5a4bfe89dc919cde35dc64, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:22.156665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:22.158252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db32b6-095d-4576-ab80-a09b82b80940 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:22.164045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 49/97] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:21 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=096974dcaa5a4bfe89dc919cde35dc64&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:24.037908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81dab-0161-4dbc-bcd9-27779d917b6f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.038405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81dab-0161-4dbc-bcd9-27779d917b6f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.038477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81dab-0161-4dbc-bcd9-27779d917b6f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.039959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 49/98] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:24.045854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad539-779b-4b40-b112-90517030a148 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.046258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad539-779b-4b40-b112-90517030a148 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.046444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad539-779b-4b40-b112-90517030a148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.070021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-801ad539-779b-4b40-b112-90517030a148 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:24.071226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-801ad539-779b-4b40-b112-90517030a148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:24.110118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:24.158998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 50/99] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:24.224511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2826a-44fb-4659-8fc8-cd8683a1e3f2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.224811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2826a-44fb-4659-8fc8-cd8683a1e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.225071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2826a-44fb-4659-8fc8-cd8683a1e3f2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.226477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 50/100] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:24.232950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.233244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.233531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.262084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:24.263335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:24.344403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e875-ccce-4a6c-af68-987f2b6ff391 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.344705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e875-ccce-4a6c-af68-987f2b6ff391 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.344860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e875-ccce-4a6c-af68-987f2b6ff391 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.346232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 51/101] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:24.351519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.351868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.352153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.373394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 51/102] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:24.381514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:24.382818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:24.460623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 52/103] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:24.631002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:24.759850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:24.764892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bfb72-4bdd-4416-b5d9-35cfef12f3f2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:24.790012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:24.807654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:24.808305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.808655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.808930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:24.826037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:24.830752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:24.831679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:24.832448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 52/104] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:24.835208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:24.957434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cde8b-f5be-4a5f-afe8-f0071a51410f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:24.997057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:24.997771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:24.998059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:24.998339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.009389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c891a0-2a2d-43d7-a429-5d15e544a3ef None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.010578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:25.015779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:25.016616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:25.017583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 53/105] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:25.018249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.035915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.039035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.050342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.050994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.051282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.051559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.052800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.054424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609e185-d037-4f6f-b5df-fff6beb9769c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.057766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 53/106] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:25.057766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bbe9-5cc7-41e8-99b4-12d6dbb13ea1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.057766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bbe9-5cc7-41e8-99b4-12d6dbb13ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.057766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bbe9-5cc7-41e8-99b4-12d6dbb13ea1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.060234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 54/107] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:25.061243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540b406-eee2-47b8-bde0-7450d87f3996 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.061582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540b406-eee2-47b8-bde0-7450d87f3996 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.061905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540b406-eee2-47b8-bde0-7450d87f3996 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.063276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 55/108] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:25.064823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540b406-eee2-47b8-bde0-7450d87f3996 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.126845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd375b-4880-44ac-afe8-caeafaa8b5c3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.160895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.161511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.161829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.162112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.162785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:25.163096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.164496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.169176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540b406-eee2-47b8-bde0-7450d87f3996 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.172901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 54/109] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:25.173705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.206010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.206619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.206908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.207186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.207903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:25.208160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.209545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e08d99-2780-4aac-a25b-b58fc96292cc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.218054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 56/110] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:25.219372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec6ae-e8cf-4996-97ab-d9aadfe482d3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.219659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec6ae-e8cf-4996-97ab-d9aadfe482d3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.219933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec6ae-e8cf-4996-97ab-d9aadfe482d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.237296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1edec6ae-e8cf-4996-97ab-d9aadfe482d3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:25.238513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1edec6ae-e8cf-4996-97ab-d9aadfe482d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:25.257565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.278196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7600-4b21-402b-9af9-574c3fb146a8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.285051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.305518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 57/111] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:25.311410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.312025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.312304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.312576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.313214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.314699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54f18-3630-433e-b46c-cfdbb6835956 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.322115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.322115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.322115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.341573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:25.342731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:25.346111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 55/112] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:41:25 2022] POST /identity/v3/projects => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:25.346111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3391c3-b920-4041-ba39-8d9e4fcebf93 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.346111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3391c3-b920-4041-ba39-8d9e4fcebf93 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.346111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3391c3-b920-4041-ba39-8d9e4fcebf93 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.346111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 56/113] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:25.359296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3391c3-b920-4041-ba39-8d9e4fcebf93 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.429099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 58/114] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:25.429974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.498369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3391c3-b920-4041-ba39-8d9e4fcebf93 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.538376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.543763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.544757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.545232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.545793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] PATH_INFO: `/v3/users/0e732c712b0843449e8ab303bbbf1a51` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.566794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] RBAC: Authorizing `identity:get_user(user_id=0e732c712b0843449e8ab303bbbf1a51)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.568900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772421a-9fef-4b76-b115-d5dc2ff0e827 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.571364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 57/115] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/users/0e732c712b0843449e8ab303bbbf1a51 => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:25.573176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.573687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.574322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.577014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef518-dd3d-441a-a1e4-4a9231a01459 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.596356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:25.597064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.597571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:25.629405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.630084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.630375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.630654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.632820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.635204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.637411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 59/116] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:25.638199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.689961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 58/117] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:25.690846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.778439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d14aa-9df3-4de3-bf3d-41da1f85692f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.805154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96febc-8b5d-4f47-80fc-a5370524d9de None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:25.824603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.825360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.825670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.825948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.832479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.838394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:25.845529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:25.846407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:25.847229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 60/118] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:25.848448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:25.853290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:25.854010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:25.854300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:25.854578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:25.892550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:25.955748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:25.957977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:25.959795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1cc9-9709-45e9-b008-036ce9b91024 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.003313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.003969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.004252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.004529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.005781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.007254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.008915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 61/119] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.009756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.076895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 59/120] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:26.076895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.097716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.120144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ff3b-e557-4c7f-ad5a-71ac6a8ec472 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.129536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.156909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.157554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.157865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.158143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.169127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:26.174315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:26.175146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:26.175926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 62/121] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:26.181308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.188883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473712a-05c5-4063-b2d7-512979099994 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.224917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.225548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.225881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.226751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.227577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:26.227894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.229274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.238874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 60/122] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.239689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.297505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035434-9836-4251-8cb5-f7fe81e90660 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.324776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.333534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.334392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.334677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.334952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/users/29b6162ed5d14f26ae87d2e2d8c81d1b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.336887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, user_id=29b6162ed5d14f26ae87d2e2d8c81d1b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.338479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.345158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59fbc-c355-4fea-90b9-9cbff5072b6e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.354624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 63/123] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:26 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/users/29b6162ed5d14f26ae87d2e2d8c81d1b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:26.355981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.386850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.387516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.387798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.388075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.400027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:26.400251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:26.406276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:26.407267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:26.408043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 61/124] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:26.413454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.511404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dd973-873f-40ab-9fa6-be7167ece32d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.544780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cb47-8c39-406e-a264-422566a54914 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.564933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.566720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.567499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.567981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.570369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.572293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.575680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 64/125] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.577028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.584177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.584740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.585023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.585299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.586007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:26.586317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.587690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.601740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 62/126] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.602880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.607634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.701766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.710042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe922e-027e-46dd-b6b1-11b4e8ddcedf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.742378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb044b28-2e66-4378-9be0-aa27485567b7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.761566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.763670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.764172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.764666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.766601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.768553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.771472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 65/127] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.772780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.782076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.782716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.783002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.783275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.784056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:26.784366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.786213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.799446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 63/128] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.800720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.874551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:26.911199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a003-d5fd-4103-8926-c0d703962c16 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.917888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a67-0cb6-4056-9bdc-6efd176075b4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:26.962391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.962719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:26.963306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.963588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.963864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.964214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:26.964604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=0e732c712b0843449e8ab303bbbf1a51) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:26.965434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.965699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:26.966389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:26.966956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.968077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:26.970455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:26.972821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 64/129] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.974013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:26.977714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 66/130] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/role_assignments?user.id=0e732c712b0843449e8ab303bbbf1a51&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:26.979264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.011809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:27.102299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63432942-8508-4631-9411-34730d4e9c09 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.113905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4252-01f6-4345-a990-d7b4d49acf10 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.149569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.151173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.152190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.153246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.155174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.156867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.159301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 67/131] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:27.160772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.170103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.170840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.171126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.171408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.178634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.183273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.187368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 65/132] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:26 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:27.189490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.200009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:27.310723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:27.320826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056fe1-0a64-4efc-affa-848403a1e6d0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.361639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.362633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.363062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.363727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] PATH_INFO: `/v3/projects/096974dcaa5a4bfe89dc919cde35dc64` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.365032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] RBAC: Authorizing `identity:get_project(project_id=096974dcaa5a4bfe89dc919cde35dc64)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.367190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.373692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 68/133] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:27 2022] GET /identity/v3/projects/096974dcaa5a4bfe89dc919cde35dc64 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:27.374771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.395825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baed04-efb2-4319-a8dc-475abe0713d9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.493082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6a267-41e1-4e88-9a1d-9782b55f13af None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.494261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.494922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.495207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.495496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.497342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.498837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.500795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 66/134] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:27 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:27.501560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.535555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.536489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.536783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.537302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/users/1058c3362abc419ba2002e2ebf5395c7/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.539979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, user_id=1058c3362abc419ba2002e2ebf5395c7, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.542431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc387504-f084-45c0-a841-706ebecb1b10 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.565016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 69/135] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:27 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/users/1058c3362abc419ba2002e2ebf5395c7/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:27.566534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2230d-1734-4074-9bcc-4af91b044c8f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.566823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2230d-1734-4074-9bcc-4af91b044c8f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.567101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2230d-1734-4074-9bcc-4af91b044c8f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.568777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 70/136] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:27.569571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2230d-1734-4074-9bcc-4af91b044c8f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:27.608528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:27.616927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:27.678060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dddeb5-cdbd-41de-a5e7-9595ce2b8a43 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.725531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2230d-1734-4074-9bcc-4af91b044c8f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:27.753265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.753942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.754231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.754513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] PATH_INFO: `/v3/projects/096974dcaa5a4bfe89dc919cde35dc64/users/1058c3362abc419ba2002e2ebf5395c7/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.757489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] RBAC: Authorizing `identity:create_grant(project_id=096974dcaa5a4bfe89dc919cde35dc64, user_id=1058c3362abc419ba2002e2ebf5395c7, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.759064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c04ad3-9617-46d6-b9f0-e2ef035ee57f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.770460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:27.771366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.771895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.772354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/users/1058c3362abc419ba2002e2ebf5395c7/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.775064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, user_id=1058c3362abc419ba2002e2ebf5395c7, role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:27.776745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:27.781632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 67/137] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:27 2022] PUT /identity/v3/projects/096974dcaa5a4bfe89dc919cde35dc64/users/1058c3362abc419ba2002e2ebf5395c7/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:27.781632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:27.781632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:27.781632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:27.804360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:27.805741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:27.805917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 71/138] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:27 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/users/1058c3362abc419ba2002e2ebf5395c7/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:27.892881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 68/139] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:28.374955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:28.484839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:28.503944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a649-ffa8-4ee3-953f-b09b78e3e383 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:28.555762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:28.556416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:28.556692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:28.556963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:28.558159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:28.559758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644351d-50c5-4cf6-ad0d-cba27c73b1b8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:28.561687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 72/140] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:28.579345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:28.675982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:28.725424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba52c2-be72-46a0-bab7-6ef74a37bce5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:28.772694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:28.773320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:28.773619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:28.773906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:28.774571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:28.776028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297df41e-7221-483f-8ebc-d42b9f2f09c1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:28.812363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 69/141] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:28 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:28.847120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd1e4-2184-467b-ad68-59a5ca0c5f4c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:28.847425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd1e4-2184-467b-ad68-59a5ca0c5f4c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:28.847691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd1e4-2184-467b-ad68-59a5ca0c5f4c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:28.849518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 73/142] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:28.855507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:28.855802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:28.856084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:28.875462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:28.876694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:28.953956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 70/143] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:29.293520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dd1e4-2184-467b-ad68-59a5ca0c5f4c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:29.418212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dd1e4-2184-467b-ad68-59a5ca0c5f4c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:29.462384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:29.463027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:29.464689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:29.465496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:29.479077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:29.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:29.485257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:29.486453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 74/144] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:29.493189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:29.547150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:29.607059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcf888-cead-496a-bdbb-2a3bc8f5d020 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:29.645150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:29.645813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:29.646103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:29.646382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:29.647070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:29.647379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:29.648769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05ac97-f455-4185-a7d2-98d9bc604861 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:29.658638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 71/145] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:29.664584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:29.698679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:29.753570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a6ce-93e5-487a-abf1-dbb424bba5de None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:29.754258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a6ce-93e5-487a-abf1-dbb424bba5de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:29.754776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a6ce-93e5-487a-abf1-dbb424bba5de None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:29.756445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 72/146] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:29.762650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:29.763194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:29.763480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:29.794895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:29.796123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:29.819056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32545-5465-4152-95a0-e581154b562f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:29.873582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 73/147] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:29.878584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:29.879608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:29.880105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:29.880573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] PATH_INFO: `/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:29.886305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] RBAC: Authorizing `identity:get_user(user_id=29b6162ed5d14f26ae87d2e2d8c81d1b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:29.888587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:29.890462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 75/148] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:29 2022] GET /identity/v3/users/29b6162ed5d14f26ae87d2e2d8c81d1b => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:29.896916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:29.946313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.012348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a64491-6444-4fa4-a01b-4f563a93fcbe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.048174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.048800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.049098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.049377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.120440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:30.122464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:30.162892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.207452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.230124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 74/149] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:30.237148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.364583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.398575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61736bf-650d-4ccd-8bdb-036d24882aab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.410204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bfac-02b6-44fa-99b6-b5f867caef70 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.438907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.439594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.439879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.440188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.441777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:30.443597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:30.445387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 75/150] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:30.451966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.462221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.488627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.489971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.490441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.490916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.514581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:30.520146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:30.520974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:30.521763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:30.524910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 76/151] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:30.531819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.575896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e994f-62f3-453d-9483-8385d2c4cd4d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.617546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.618211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.618484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.618764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.619683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=29b6162ed5d14f26ae87d2e2d8c81d1b) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:30.620003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:30.622618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270f188-04a7-4129-94c8-ca6740f89f9a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:30.630275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 76/152] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/role_assignments?user.id=29b6162ed5d14f26ae87d2e2d8c81d1b&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:30.634851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf21f20-f3f0-4511-952c-19eb43bca4e8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.635137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf21f20-f3f0-4511-952c-19eb43bca4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.635416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf21f20-f3f0-4511-952c-19eb43bca4e8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.637899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 77/153] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:30.644140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.644438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.644702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.668822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b890e-2b55-4db0-8479-03839787a23a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.675446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:30.676669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:30.707781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.709176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.709763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.710278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.711215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:30.711722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:30.713416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:30.723824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 77/154] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:30.729918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.740838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.787287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:30.798961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 78/155] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:30.849430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7e19-2147-46d7-8b37-9dd46cba8c40 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:30.886135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:30.887523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.887862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.888135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] PATH_INFO: `/v3/users/0e732c712b0843449e8ab303bbbf1a51` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.889347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] RBAC: Authorizing `identity:get_user(user_id=0e732c712b0843449e8ab303bbbf1a51)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:30.890881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c932422-d3bd-4604-8610-e91eea6c33cb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:30.892589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 78/156] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/users/0e732c712b0843449e8ab303bbbf1a51 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:30.898181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:30.953424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d541fa-2358-48c2-9cc0-dad67ddfb647 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.953740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d541fa-2358-48c2-9cc0-dad67ddfb647 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.953952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d541fa-2358-48c2-9cc0-dad67ddfb647 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.955524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 79/157] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:30.961125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:30.961405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:30.961719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:30.980837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:30.982119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:31.040961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287d734-6030-438c-b9b7-60a7fd0f4500 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:31.053522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 80/158] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:31.054339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:31.091817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:31.092468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.092752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.093032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.094200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:31.095666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:31.097716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 79/159] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:30 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:31.103482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:31.213235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6097e-0bb8-4e68-ac68-41abc9dc2272 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:31.249980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:31.250647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.250928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.251201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/users/0e732c712b0843449e8ab303bbbf1a51/roles/4f33402556ca4c3d8dcafd31f341a974` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.253133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, user_id=0e732c712b0843449e8ab303bbbf1a51, role_id=4f33402556ca4c3d8dcafd31f341a974)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:31.254617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:31.278488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 81/160] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:31 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/users/0e732c712b0843449e8ab303bbbf1a51/roles/4f33402556ca4c3d8dcafd31f341a974 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:31.407526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:31.491087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:31.504667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:31.537471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13f15-5307-4c6a-8364-a568267cc6a7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:31.608282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:31.609255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:31.609933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.610216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.610498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.635126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:31.645277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:31.646233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:31.646979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:31.648528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 80/161] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:31.658842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:31.692994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1f28e-f31a-4851-ab49-3f6c22d1a9e5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:31.787364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:31.788025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.788308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.788578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.818575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e75bf42c-bbdc-4da7-af3b-5250500fb949 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:31.823913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:31.825369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:31.826604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 82/162] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:31.832515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9228-24f8-44da-992f-547997e1fce5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.832801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9228-24f8-44da-992f-547997e1fce5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.833081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9228-24f8-44da-992f-547997e1fce5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.834658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 83/163] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:31.835378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9228-24f8-44da-992f-547997e1fce5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:31.887947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13e9fc-dad1-4927-8580-f4bc38cb1962 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:31.947887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:31.972605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:31.973237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:31.973517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:31.973815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:31.974703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:31.975015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:31.976420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72b4c3-102c-4ceb-b66b-cf2fd17a2938 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:32.000504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 81/164] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:32.001509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:32.001824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:32.002104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:32.046202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:32.047387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:32.060460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9228-24f8-44da-992f-547997e1fce5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:32.095290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:32.125076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:32.125781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:32.126065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:32.126354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:32.127015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:32.127322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:32.129116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:32.147835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 84/165] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:32.148721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:32.181507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 82/166] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:32.182165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:32.364979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905dd31-227a-43a9-b6a4-60443c7128cd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:32.394155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e68ac-d156-46d8-867c-7e2ad3c0e486 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:32.418026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:32.430971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:32.431623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:32.431904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:32.432181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:32.433317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:32.435434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:32.437008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 83/167] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:32 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:32.442688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:32.450972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:32.451641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:32.451929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:32.452209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:32.453379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:32.454902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:32.456799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 85/168] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:32 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:32.470258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:32.569617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274b08e-f5a1-45a1-aa54-c13f8a9e1fca None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:32.624615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:32.625462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:32.625955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:32.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:32.682697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:32.714159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:32.733022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:32.734963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:32.877786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 84/169] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:32.878742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:32.904133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129841-46ce-409d-b7bf-78614e86c901 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:32.982024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:32.998030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc090cfe-a555-40cc-b773-445bda4d6b18 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.026605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.028128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.028478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.028824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.038238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.038873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.039150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.039418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.052002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:33.057117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:33.058056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:33.059400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 85/170] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:33.060073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:33.218720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.220529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.263414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7055-3f18-49d3-bc57-a3759d330374 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.347213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:33.352678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.353333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.353627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.355063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.358384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.359863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.361543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 86/171] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:33.362428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:33.401956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:33.471128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 86/172] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1001 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:33.474400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:33.539108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b604-75da-4de4-b7a6-14b8c6afcc14 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.583665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.584995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.585463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.585881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.586967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:33.587272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.588808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9972f-0efb-4cd0-9c62-7ba978e212f9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.600111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 87/173] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:33.605807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3be0c-c7fb-4fd5-a933-ace1e5c0e3f1 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.605807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3be0c-c7fb-4fd5-a933-ace1e5c0e3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.605807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3be0c-c7fb-4fd5-a933-ace1e5c0e3f1 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.605807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 88/174] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:33.605807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3be0c-c7fb-4fd5-a933-ace1e5c0e3f1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:33.642391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a2ba9-124d-4302-b4aa-11aaf4722d7a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.687415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.688996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.690026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.690503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.691462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:33.691978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.694168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81521f-7830-4fd5-b569-b6f441d20a07 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.703085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 87/175] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:33.704737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c604ac-7003-4cf0-8606-216d22cb4a1f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.705224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c604ac-7003-4cf0-8606-216d22cb4a1f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.705918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c604ac-7003-4cf0-8606-216d22cb4a1f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.707786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 88/176] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:33.708914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c604ac-7003-4cf0-8606-216d22cb4a1f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:33.728082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:33.736008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:33.766394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3be0c-c7fb-4fd5-a933-ace1e5c0e3f1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.820534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.821625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.822230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.822675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.824957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.827264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113928b5-2412-43ee-b8d3-6e2d19cd119b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.829559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 89/177] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:33.832272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19915-b794-4bf9-b67f-550a2e89d16b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.833014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19915-b794-4bf9-b67f-550a2e89d16b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.833440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19915-b794-4bf9-b67f-550a2e89d16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.845972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c604ac-7003-4cf0-8606-216d22cb4a1f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:33.854941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-60b19915-b794-4bf9-b67f-550a2e89d16b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:33.857030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-60b19915-b794-4bf9-b67f-550a2e89d16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:33.896404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:33.897057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.897436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.897753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] PATH_INFO: `/v3/users/1058c3362abc419ba2002e2ebf5395c7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.899825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] RBAC: Authorizing `identity:get_user(user_id=1058c3362abc419ba2002e2ebf5395c7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:33.901356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6eee39-3e90-45ee-845c-60d796b442e1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:33.903932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 89/178] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/users/1058c3362abc419ba2002e2ebf5395c7 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:33.904965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.905251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.905524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.927973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:33.929178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:33.943763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 90/179] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:33.951092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623668-1903-4190-bc0a-585fa09f5743 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:33.951381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623668-1903-4190-bc0a-585fa09f5743 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:33.952008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623668-1903-4190-bc0a-585fa09f5743 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:33.953470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 91/180] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:33.956937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b623668-1903-4190-bc0a-585fa09f5743 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.018629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 90/181] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:34.019938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.037978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.057267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.089285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b623668-1903-4190-bc0a-585fa09f5743 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.127025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.127644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.127923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.128197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.128934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:34.129248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.130827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78127a-927b-4cee-927a-02df246e02d8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.137422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 92/182] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:33 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:34.138147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e23ba-d7f6-4e2c-b7dd-d7ff93095159 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.138699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353c06c-890f-4a54-a4fa-ab6d45d1758f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.138986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353c06c-890f-4a54-a4fa-ab6d45d1758f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.139267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353c06c-890f-4a54-a4fa-ab6d45d1758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.160789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b353c06c-890f-4a54-a4fa-ab6d45d1758f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:34.162098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b353c06c-890f-4a54-a4fa-ab6d45d1758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:34.180995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.182110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.182648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.183276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.259648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 93/183] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:34.260740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9db97-e20e-404d-9464-1b801ee6a352 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.261023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9db97-e20e-404d-9464-1b801ee6a352 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.261295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9db97-e20e-404d-9464-1b801ee6a352 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.265769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 94/184] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:34.268417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.268760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.268984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.272366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.274242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.289700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:34.290917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:34.297062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.367797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.374915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 95/185] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:34.375764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.417376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 91/186] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:34.422789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.493376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b54aa-80b4-4e37-94d5-4909317d9b0b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.532741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.533871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.535306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.535759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.549713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1db51e-7d3c-49c9-9e37-d3d86c208600 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.558883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:34.565327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:34.566270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:34.567030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 96/187] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:34.567676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.568720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.610107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.610974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.611410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.611829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.624098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:34.629286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:34.630175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:34.631047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 92/188] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:34.631717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.653163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.701445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea5fd-8c98-4337-b0ba-225d16f34ffe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.736781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc439c13-0f61-4349-8f6b-0cdd09bb73fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.737381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.737907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.738212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.738488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] PATH_INFO: `/v3/projects/096974dcaa5a4bfe89dc919cde35dc64` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.739627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] RBAC: Authorizing `identity:get_project(project_id=096974dcaa5a4bfe89dc919cde35dc64)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.741067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.742719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 97/189] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/projects/096974dcaa5a4bfe89dc919cde35dc64 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:34.743479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.776290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.776898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.777487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.777792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.778452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:34.778752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.780135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.788818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 93/190] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:34.789541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.825212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.844506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc5f99-eeb0-4c0d-a8c4-458d32f2cd44 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.869849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:34.878959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.879586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.879863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.880133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.881235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.882709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.884267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 98/191] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:34.885021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.893837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4b74-278d-4735-9853-be3efde6f570 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:34.928719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:34.929324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:34.929627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:34.929903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:34.930595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:34.930901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:34.932291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:34.940037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 94/192] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:34.942471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:34.986907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703164a-ccd7-4fee-92d5-b9c7263a2fca None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.020817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.021453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.021772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.022050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.033040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:35.036174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:35.037797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:35.038670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:35.039854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 99/193] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:35.040755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.045276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c6a4-de38-4634-9ec0-b6ac0c15b2ea None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.079230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.079838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.080121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.080395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.081117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=096974dcaa5a4bfe89dc919cde35dc64, user.id=1058c3362abc419ba2002e2ebf5395c7) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:35.081421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.082903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.088678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 95/194] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:34 2022] GET /identity/v3/role_assignments?user.id=1058c3362abc419ba2002e2ebf5395c7&scope.project.id=096974dcaa5a4bfe89dc919cde35dc64&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:35.113756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.113907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.156269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8f585-659d-486a-b641-e451577db2fb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.198587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.199775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.200060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.200335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.205374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2280cb6-9ef8-44c4-9415-3a7034ff6c23 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.216377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.218079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.224606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 100/195] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:35.224606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.251562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.252182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.252475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.252742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.253393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.254953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.282613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 96/196] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:35 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:35.283744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.306699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.364261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeab687-3561-470e-b8eb-658a5c8ed2de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.380324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.403709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.404363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.404649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.404988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.406181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.407659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.409323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 101/197] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:35.410146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.424423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc790d-b174-42fd-a963-5c029a7b9ab8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.467367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.467989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.468268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.468544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.469219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:35.469522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.471072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.481906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 97/198] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:35.483242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.525339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7486f-9cb6-4da7-9fcb-b40fb762c8ab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.573157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.573777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.574619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.574909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.578983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.608002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d96d-d42a-4e7f-8fe4-52f80d4628a8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.635916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.651337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.652314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.652674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.653021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.659287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.660793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.738465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.740273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.803788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 102/199] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:35.805610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.844180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.874504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 98/200] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:35.875871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.909197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:35.928339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da50bf-258f-4707-8a0a-d2c67ccfe4bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:35.965331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:35.966010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:35.966290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:35.966590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] PATH_INFO: `/v3/users/0e732c712b0843449e8ab303bbbf1a51` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:35.968200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0e732c712b0843449e8ab303bbbf1a51)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:35.969775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:35.971764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 103/201] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/users/0e732c712b0843449e8ab303bbbf1a51 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:35.972550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:35.999748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12a7f-2aa3-48dc-9a93-d73cc3d77653 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.038818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.039533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.039814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.040091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.041432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.043460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.045279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 99/202] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:36.046095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:36.080900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df04bc7-247d-4892-ad62-9fe522f400a4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.097644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:36.118039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.118687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.118972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.119251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.120546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.122423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.128910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 104/203] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:35 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:36.128910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:36.158498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:36.168843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f06e-f3a5-46d2-8ea7-97e0cfa8d704 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.208477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.208997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.209275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.209548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.260136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155ad5e-0d0b-4071-93e3-ed94c27480be None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.297308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.297957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.298292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.298563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.298990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.299334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=f78d8abdbccd40fea4df8e087996d43e) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:36.299641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.301058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.301278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.307091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 105/204] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:36 2022] GET /identity/v3/role_assignments?user.id=f78d8abdbccd40fea4df8e087996d43e&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:36.307884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:36.361410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:36.452708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 100/205] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:36.459415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:36.461041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:36.477723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c35a3-d445-4be9-bbbc-2bec87fcc961 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.554270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.555128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.555546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.556038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.556919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=f78d8abdbccd40fea4df8e087996d43e) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:36.558720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.560324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.568746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 106/206] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:36 2022] GET /identity/v3/role_assignments?user.id=f78d8abdbccd40fea4df8e087996d43e&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:36.639976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c4b-2624-4d54-b1ab-64232fceb4cf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.698351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.699721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.700099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.700472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.701888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.703549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6830c8-2984-4da8-92a9-15bcfce47e1c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.705878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 101/207] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:36 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:36.711580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:36.834558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85248fc-4e14-4ef7-8699-3c71390bee63 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:36.875086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:36.875989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:36.876438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:36.876878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:36.877824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, scope.project.id=995426667c154ea2a375fda95b1f26ff, user.id=0e732c712b0843449e8ab303bbbf1a51) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:36.878307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:36.879989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270546fa-0acc-438a-b8ad-04150221610a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:36.890428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 107/208] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:36 2022] GET /identity/v3/role_assignments?user.id=0e732c712b0843449e8ab303bbbf1a51&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:37.523402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201072-8f8e-4ee4-a238-6616c22ec94a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:37.523926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201072-8f8e-4ee4-a238-6616c22ec94a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:37.523926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201072-8f8e-4ee4-a238-6616c22ec94a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:37.525633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 102/209] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:37.531635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7745-cd93-4be9-ad49-9c609377aa17 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:37.532011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7745-cd93-4be9-ad49-9c609377aa17 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:37.532366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7745-cd93-4be9-ad49-9c609377aa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:37.552112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a84c7745-cd93-4be9-ad49-9c609377aa17 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:37.553322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a84c7745-cd93-4be9-ad49-9c609377aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:37.557343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:37.640812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 108/210] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:37.741332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d39d-b634-4acb-b5a2-1aa0e56e1cfd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:37.741638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d39d-b634-4acb-b5a2-1aa0e56e1cfd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:37.741925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d39d-b634-4acb-b5a2-1aa0e56e1cfd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:37.743326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 103/211] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:37.748620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:37.748910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:37.749191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:37.767418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:37.768960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:37.840835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 109/212] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:37.916396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0d39d-b634-4acb-b5a2-1aa0e56e1cfd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:37.933426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.026309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0d39d-b634-4acb-b5a2-1aa0e56e1cfd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.062972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.063603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.063877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.064146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.075300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:38.079957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:38.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:38.081771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 104/213] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:38.091346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.119967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.166498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.205617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62b7f-01cf-4fbd-9469-b794e9189df1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.220143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.247640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.248744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.249265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.249799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.250770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:38.251364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:38.252985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:38.254279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372bdb4-4b67-4a04-a229-6f198ffef364 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.268965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 110/214] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:38.270433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.299137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.300205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.300500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.300781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.302220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:38.304364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:38.306469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 105/215] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:38.312398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.388376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90264a7-7c78-4fd2-90f7-5c4f0ff6b084 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.415909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea984e-39d0-42e2-9f22-f9b664bf154b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.423002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.430736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.431786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.432269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.432743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] PATH_INFO: `/v3/users/b534e4e8ee27465bbfa8ba4626905c1a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.448250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] RBAC: Authorizing `identity:get_user(user_id=b534e4e8ee27465bbfa8ba4626905c1a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:38.450069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:38.451428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.452045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.452326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.452602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.457637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:38.457637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f278b-745d-445b-8e23-2724dcd05182 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:38.458130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 111/216] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/users/b534e4e8ee27465bbfa8ba4626905c1a => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:38.459094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.475551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.489926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 106/217] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:38 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:41:38.543851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17508a94-5ba9-4794-bf66-b5b4d063daa0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.544507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17508a94-5ba9-4794-bf66-b5b4d063daa0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.544507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17508a94-5ba9-4794-bf66-b5b4d063daa0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.545785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 107/218] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:38.551258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc20e-02df-446b-8296-0d0748f9aa00 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.551565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc20e-02df-446b-8296-0d0748f9aa00 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.551806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc20e-02df-446b-8296-0d0748f9aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.571282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-772dc20e-02df-446b-8296-0d0748f9aa00 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:38.572499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-772dc20e-02df-446b-8296-0d0748f9aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:38.587438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d765-904f-49ba-b1b9-50aca59c1c96 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:38.630546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:38.632362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.632674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.633013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.648390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 108/219] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:38.649494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa874b8-2e92-43d9-a615-f17183612444 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.649821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa874b8-2e92-43d9-a615-f17183612444 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.650093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa874b8-2e92-43d9-a615-f17183612444 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.651361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 109/220] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:38.657124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:38.657409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:38.657724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:38.676445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:38.677647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:38.706817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.729150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:38.730879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:38.763480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 110/221] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:38.863283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 112/222] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:38.869042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.929163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:38.936185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:38.987903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d550bb-5cfc-4e1a-82c3-bc999903612a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.048277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7cef2-51e0-4e28-96dc-b18b3f4bd77c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.050322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.051014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.051296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.051571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.052480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.052798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.054411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.056184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 111/223] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.058017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.097058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.097795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.098079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.098407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.113933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:39.119140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:39.120021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:39.120827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:39.123524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 113/224] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:39.124530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.190566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4656-c6d3-453c-80b3-aacd318406b0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.226747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.229382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c6b9-020a-45f8-a181-44f01c03389d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.239600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.240259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.240543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.240818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.255419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:39.263328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:39.264247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 112/225] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:39.265731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.273422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.274344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.274847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.275327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.276376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=b534e4e8ee27465bbfa8ba4626905c1a) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:39.276896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.279087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.287970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 114/226] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/role_assignments?user.id=b534e4e8ee27465bbfa8ba4626905c1a&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.289362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.336038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.390931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2e18-debf-4749-884c-ca0e7123edc1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.437212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.438253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.438730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.439214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.440087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:39.440637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.442270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.459652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 113/227] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.466072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.525358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea718b3-928d-4991-ba6b-f606afc9dee7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.566533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.594969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.595502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.595794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.596074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.597455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:39.597455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.598585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.617060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 115/228] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.622477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.629753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50225-6e8a-40d4-9715-2e6c783eb709 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.666353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.667369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.668386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.669783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.671467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.673631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 114/229] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.679128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.778363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.821825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ee40-02c5-4316-a405-f6416809b148 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.839277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b05184-b504-4dbf-963d-12ef00ce2eaf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:39.857331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:39.876650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.877709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.878895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.878895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.879811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.881374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.883333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 115/230] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:39.889532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:39.890129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:39.890916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:39.891344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:39.891773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:39.898361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:39.900048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:39.902708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 116/231] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:39.907845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:40.001118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91566e-866e-4a30-af0e-39380266f873 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:40.046621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:40.049112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:40.049540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:40.050994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/f78d8abdbccd40fea4df8e087996d43e/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:40.054927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, user_id=f78d8abdbccd40fea4df8e087996d43e, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:40.058476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:40.088983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 116/232] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:39 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/f78d8abdbccd40fea4df8e087996d43e/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:41:40.130851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78719df-bd48-4b3c-a424-2fe678ed0f9e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:40.172752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:40.173661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:40.174117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:40.174537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:40.178325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:40.179963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae2710-1b0e-4a11-b6ad-daee0c326ed1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:40.182216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 117/233] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:39 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:40.187812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:40.249259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:40.324266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d43f-c530-4274-a76f-dfd1247b433c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:40.376732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:40.377418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:40.377729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:40.378009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/f78d8abdbccd40fea4df8e087996d43e/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:40.383642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, user_id=f78d8abdbccd40fea4df8e087996d43e, role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:40.385166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0430ea-8a9b-4fbb-9f52-5e56f51a9557 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:40.415199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 117/234] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:40 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/f78d8abdbccd40fea4df8e087996d43e/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:40.968697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec004e-ceb8-48be-b179-451292ab4082 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:40.969574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec004e-ceb8-48be-b179-451292ab4082 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:40.970055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec004e-ceb8-48be-b179-451292ab4082 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:40.972126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 118/235] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:40.978042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5a50-3907-49e8-8a65-e1fac4c31f53 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:40.979035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5a50-3907-49e8-8a65-e1fac4c31f53 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:40.979492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5a50-3907-49e8-8a65-e1fac4c31f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.001717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f66f5a50-3907-49e8-8a65-e1fac4c31f53 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:41.003502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f66f5a50-3907-49e8-8a65-e1fac4c31f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:41.099181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 118/236] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:41.131690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd5b0-44fc-4739-a17e-1f13fd6454c3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.131976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd5b0-44fc-4739-a17e-1f13fd6454c3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.132258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd5b0-44fc-4739-a17e-1f13fd6454c3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.133909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 119/237] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:41.139625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.139913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.140188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.160032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:41.161225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:41.209968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:41.264253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 119/238] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:41.425980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbd5b0-44fc-4739-a17e-1f13fd6454c3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:41.445917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:41.563278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbd5b0-44fc-4739-a17e-1f13fd6454c3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:41.603625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:41.604272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.604550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.604824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.606662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:41.608147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:41.609794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 120/239] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:41.615743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:41.734232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83315a-f98d-4696-8880-df4292cc7bf3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:41.758114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:41.780398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:41.781720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.782208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.782684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.783530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:41.786011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54abe1-c882-4c14-97f2-ca18ccd28216 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:41.823151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 120/240] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:41 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:41.825354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a8383-2326-4644-89e1-6c945acb7842 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.825981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a8383-2326-4644-89e1-6c945acb7842 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.826395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a8383-2326-4644-89e1-6c945acb7842 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.828396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 121/241] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:41.838787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.839481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.840563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:41.884163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:41.891086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:41.894573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:41.919005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:41.938043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79190ef-13a5-4ffd-abd6-4b1e64d77473 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:41.994554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:41.995174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:41.995458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:41.995751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.003092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 122/242] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:42.012705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:42.017903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:42.018781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:42.019548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 121/243] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:42.026283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.134279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29306a90-3c2b-4e62-9e52-2c8847242964 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.170029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.171004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.171494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.171956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.172815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:42.173306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:42.174906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:42.186293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 123/244] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:42.192136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.298139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0415f-746a-4a5d-80d2-ca715c38c94e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.325303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.337201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.338234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.339051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.340000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] PATH_INFO: `/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.353699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.358523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.358995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] RBAC: Authorizing `identity:get_user(user_id=73c8de8efaad4ab9b8fc43a382a1e8e4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:42.361489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:42.364679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 122/245] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:42.372751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.470596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf819ce-cd35-433a-999d-a747909b94a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.505691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4c51d-b6c5-4926-87cc-a59d61d718f3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.513456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.514482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.514941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.515905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.529445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:42.535188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:42.536129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:42.536987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:42.539268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 124/246] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:42.548874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.554775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.555248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.555541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.555821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.633916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.638059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:42.639622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.639776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53673b01-4975-4bff-bb56-7cf76eb432a9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:42.667257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a1550-50a2-481e-a22a-340cd63d89f3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.712734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.713735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.714222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.715271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.716182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:42.716712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:42.719520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae0179-4a8f-45b9-b824-e27bdb76a640 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:42.729007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 125/247] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:42.730376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb479df-3301-4ad7-bc79-d7110b135176 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.730766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb479df-3301-4ad7-bc79-d7110b135176 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.731094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb479df-3301-4ad7-bc79-d7110b135176 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.732407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 126/248] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:42.738331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb479df-3301-4ad7-bc79-d7110b135176 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.770819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 123/249] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:42.775048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9b730-10cd-4462-bb0f-3e412293e0fa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.775641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9b730-10cd-4462-bb0f-3e412293e0fa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.775933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9b730-10cd-4462-bb0f-3e412293e0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.794460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-08e9b730-10cd-4462-bb0f-3e412293e0fa None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:42.795753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-08e9b730-10cd-4462-bb0f-3e412293e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:42.881914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb479df-3301-4ad7-bc79-d7110b135176 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:42.884301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 124/250] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:42.886904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a707-839e-4d13-8ae7-b2a78f86f66c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.887335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a707-839e-4d13-8ae7-b2a78f86f66c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.887617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8a707-839e-4d13-8ae7-b2a78f86f66c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.889481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 125/251] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:42.890109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8a707-839e-4d13-8ae7-b2a78f86f66c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:42.929239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:42.929658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.929866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.930103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] PATH_INFO: `/v3/users/b534e4e8ee27465bbfa8ba4626905c1a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.931308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] RBAC: Authorizing `identity:get_user(user_id=b534e4e8ee27465bbfa8ba4626905c1a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:42.932795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3680e09-924b-4709-8186-7715b4fb9069 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:42.934839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 127/252] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/users/b534e4e8ee27465bbfa8ba4626905c1a => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:42.938386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.940847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:42.941268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:42.941570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:42.959086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:42.984885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:42.986048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:43.062931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8a707-839e-4d13-8ae7-b2a78f86f66c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.100918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.101574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.101885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.102167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.103314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:43.104846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:43.107171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 126/253] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:42 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:43.107935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.118037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 128/254] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:43.118745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.212011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:43.218927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b3c98-e329-492a-be33-aaab27f1a46e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.264615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.265187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.265471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.265819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.267046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:43.268506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:43.270364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 127/255] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:43.276063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.313152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2b028-947c-473b-bcc4-79ec002e4972 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.384759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ad28-823a-4253-9c48-12ae1fd84a31 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.387167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:43.402632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.403271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.403555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.403833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.404578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=73c8de8efaad4ab9b8fc43a382a1e8e4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:43.404893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:43.406479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:43.423565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.424487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.424596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 129/256] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/role_assignments?user.id=73c8de8efaad4ab9b8fc43a382a1e8e4&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:43.424697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.424770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/b534e4e8ee27465bbfa8ba4626905c1a/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.425239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.426901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, user_id=b534e4e8ee27465bbfa8ba4626905c1a, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:43.428460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:43.449833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 128/257] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:43 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/b534e4e8ee27465bbfa8ba4626905c1a/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:43.556834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d17b-6d97-470d-8c88-d85eaab4d4e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.567485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.599272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.599950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.600234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.600511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.614815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:43.620646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:43.622123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:43.622925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 130/258] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:43.630467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.651174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:43.738391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:43.800663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163fc3-dc49-4b07-89a2-aa9ce083c48a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.833655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4259-9b53-4b20-a03a-de1d5e3d6641 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:43.877035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.877882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.878159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.878425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.896307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:43.901552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:43.903000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:43.903786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:43.909154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 129/259] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:43.910751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:43.911384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:43.911658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:43.911927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:43.912606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:43.912905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:43.914326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:43.918772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:43.928372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 131/260] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:43.938647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:44.119388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:44.140380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b20a2-26f1-4c7d-b9ce-4db61c55d93b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:44.142743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378d89-5d5e-44e9-919a-43255ddf10d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:44.172080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:44.183995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:44.185003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:44.186234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:44.187259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:44.188225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:44.189587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:44.191157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:44.207224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 130/261] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:44.212402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:44.213214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:44.213650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:44.214071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:44.216517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:44.216868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:44.218842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:44.224153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 132/262] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:43 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:44.231620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:44.368504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c734-799b-47bb-862b-732d17343598 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:44.405842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:44.406818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:44.407290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:44.407752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] PATH_INFO: `/v3/users/f78d8abdbccd40fea4df8e087996d43e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:44.409105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] RBAC: Authorizing `identity:get_user(user_id=f78d8abdbccd40fea4df8e087996d43e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:44.410973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:44.416854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2d64b-2617-41ab-8ab2-c51c0832944e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:44.417220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 131/263] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:44 2022] GET /identity/v3/users/f78d8abdbccd40fea4df8e087996d43e => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:44.423234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:44.467970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:44.503827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:44.504802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:44.505272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:44.505780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:44.627979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d0bbf-ba51-47b2-8805-d0738298a90a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:44.637062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:44.676141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:44.677985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:44.686223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:44.686998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:44.687288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:44.687566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:44.830024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:44.831507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146019e2-3d9a-4393-9abf-d7d9e5f09cf7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:44.860379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:44.872826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 133/264] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:44.878625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:44.993128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:45.006047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3dfa-8b74-4235-b0e7-5d29a0bdf3e6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:45.045734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:45.046382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.046663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.046939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.048827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:45.050529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c9d25-5344-45f4-8b94-51a40f441217 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:45.055388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 134/265] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:44 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:45.056441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eb0ae-9e00-4a8e-a3a9-95bced536411 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.056727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eb0ae-9e00-4a8e-a3a9-95bced536411 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.057006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eb0ae-9e00-4a8e-a3a9-95bced536411 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.063378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 135/266] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:45.064729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38eb0ae-9e00-4a8e-a3a9-95bced536411 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:45.073269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 132/267] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:45.074990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.075278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.075557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.117237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:45.120635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:45.263705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 133/268] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:45.264499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:45.266033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38eb0ae-9e00-4a8e-a3a9-95bced536411 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:45.332115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:45.332735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.333015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.333308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.334170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=f78d8abdbccd40fea4df8e087996d43e) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:45.334472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:45.335929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce556742-c8d0-4ecb-b97b-ad85721b41f0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:45.344224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:45.348145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 136/269] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:45 2022] GET /identity/v3/role_assignments?user.id=f78d8abdbccd40fea4df8e087996d43e&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:45.389130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078a04-ec60-4f6b-b5a3-ac3121f439b6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.389438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078a04-ec60-4f6b-b5a3-ac3121f439b6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.389701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b078a04-ec60-4f6b-b5a3-ac3121f439b6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.391179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 137/270] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:45.396231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.396514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.396779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.423222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:45.428794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:45.450328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5fb5a-08a6-47fe-8924-7b9f68721e3b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:45.456379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:45.508023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:45.508733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.509018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.509291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.514463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:45.516036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:45.517813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 134/271] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:45 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:45.523901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:45.599289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 138/272] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:45.633632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:45.662274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e15b-06ad-40cf-92b3-0cc67ea91476 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:45.677483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:45.706754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:45.707370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.707650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.707921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.708661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=f78d8abdbccd40fea4df8e087996d43e) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:45.708971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:45.719271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:45.732166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 135/273] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:45 2022] GET /identity/v3/role_assignments?user.id=f78d8abdbccd40fea4df8e087996d43e&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:45.801274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682ef2c-f495-42d9-9e3e-cfc5135ff02a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:45.844404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:45.845060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:45.845337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:45.845623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:45.846766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:45.848592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:45.850527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 139/274] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:45.856452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:45.993749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2e18-5335-4efb-821a-59e7a7eb0fe4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.039039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.039668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.039956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.040236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.040898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:46.042389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758472-e3e1-4af1-aac6-f1df9ea4f13b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:46.078411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 136/275] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:41:45 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:46.091224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.114002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:46.219056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df59669-405f-49b5-8ce0-a98d3ed723d0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.238901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e082-1792-4523-b8ed-8f10f36c2efa None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.239664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e082-1792-4523-b8ed-8f10f36c2efa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.240018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e082-1792-4523-b8ed-8f10f36c2efa None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.244718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 137/276] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:46.251786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639b25-7d5c-4d14-b190-18a8c2ee2829 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.252303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639b25-7d5c-4d14-b190-18a8c2ee2829 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.252773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639b25-7d5c-4d14-b190-18a8c2ee2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.260072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.260934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.261916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.262191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.273895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:46.275459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ad639b25-7d5c-4d14-b190-18a8c2ee2829 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:46.276714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ad639b25-7d5c-4d14-b190-18a8c2ee2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4cef917a-89d3-4460-af9c-53f95ebaa041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:46.279904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:46.281028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:46.282091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 140/277] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:46.282910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05354-4ed4-409e-892c-f715937e7561 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.282910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05354-4ed4-409e-892c-f715937e7561 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.282910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05354-4ed4-409e-892c-f715937e7561 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.285639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 141/278] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:46.288933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05354-4ed4-409e-892c-f715937e7561 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.307765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:46.372265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 138/279] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:46.373489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.373881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.374149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.384637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:46.398617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:46.399881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:46.410284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05354-4ed4-409e-892c-f715937e7561 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.451975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.452509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.452769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.453039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.453742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:46.454046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:46.455516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:46.465994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 142/280] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:46.473953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.482757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 139/281] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:46.583715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c079-170c-4313-a88f-f35343bf0614 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.619376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.620030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.620354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.621165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] PATH_INFO: `/v3/users/a228897daf7a43b29be67d2cb12e3b11` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.630003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:46.637219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] RBAC: Authorizing `identity:get_user(user_id=a228897daf7a43b29be67d2cb12e3b11)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:46.639177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:46.641224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 143/282] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/users/a228897daf7a43b29be67d2cb12e3b11 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:46.648417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.677094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.807189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:46.819620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682f5bc-bd13-4f71-9214-12b285a80758 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.820163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2474ad-daf6-4b26-a1c8-ee1125993804 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:46.860031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.860668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.860941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.861213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.863108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:46.863987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:46.864285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:46.864560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:46.865294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, user.id=a228897daf7a43b29be67d2cb12e3b11, scope.system=all) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:46.865618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:46.867366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:46.872476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:46.873890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 140/283] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/role_assignments?user.id=a228897daf7a43b29be67d2cb12e3b11&scope.system=all&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:46.874759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:46.878192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:46.879119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:46.879915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 144/284] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:46.885817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.002030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:47.054816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691846a0-2eee-4c49-aba8-c426401bf6bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.060123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29a377-bd54-48cc-9bc0-ffd2a4e2fff0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.116462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.117081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.117360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.117655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.118344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.118981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.119254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.119520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.120666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:47.120980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.122628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.140921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:47.150224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 145/285] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:47.151161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:47.151608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:47.153105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:47.154477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 141/286] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:47.162791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.200970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:47.324167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f960e7f-3bc4-41f8-bd66-afdfc7eda74f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.345783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1252-ef5a-420f-816d-9cdb9f4acfaf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.357740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:47.388198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.388847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.389124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.389394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] PATH_INFO: `/v3/users/b534e4e8ee27465bbfa8ba4626905c1a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.390680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] RBAC: Authorizing `identity:get_user(user_id=b534e4e8ee27465bbfa8ba4626905c1a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.392205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.393945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 146/287] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/users/b534e4e8ee27465bbfa8ba4626905c1a => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:47.404647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.425553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.426572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.426855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.427131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.427800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:47.428104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.429476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.445568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 142/288] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:47.451882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.614792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a2134-623c-4238-8dca-e8341616d93d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.616613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:47.663845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554513-da89-44af-aef7-b5320e1bb89f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.687008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.692477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.692477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.692477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.736192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.736901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.737189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.737520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] PATH_INFO: `/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.740403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] RBAC: Authorizing `identity:get_user(user_id=73c8de8efaad4ab9b8fc43a382a1e8e4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.741885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.747616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 143/289] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4 => generated 297 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:47.749629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.804505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:47.838363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.840188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.898866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318335-39bb-44c2-b675-de47d9a4e24f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:47.958420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:47.959069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:47.959508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:47.959946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:47.963947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:47.966168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:47.968302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 144/290] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:47.975194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.984159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 147/291] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:47.994737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:47.997377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:48.155659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:48.167362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe94137-86a4-42ba-8e7f-d5c3c87ebfdc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:48.179811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65233a50-c2b9-4619-ac38-6a8da03ff8c5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:48.207183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:48.208533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.209025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.209498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/73c8de8efaad4ab9b8fc43a382a1e8e4/roles/4f33402556ca4c3d8dcafd31f341a974` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.211686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, user_id=73c8de8efaad4ab9b8fc43a382a1e8e4, role_id=4f33402556ca4c3d8dcafd31f341a974)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:48.213329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:48.229310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 145/292] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:41:47 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/users/73c8de8efaad4ab9b8fc43a382a1e8e4/roles/4f33402556ca4c3d8dcafd31f341a974 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:48.257017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:48.258147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.258683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.259238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.260636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:48.263134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dced615-126e-45cc-8888-cd408dfde893 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:48.265254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 148/293] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:47 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:48.271845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:48.358790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:48.389276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626121d5-d11f-4736-92cd-525d8ee5016f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:48.428016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:48.428989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.429499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.430005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.431053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=b534e4e8ee27465bbfa8ba4626905c1a) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:48.431652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:48.433312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365d051-2154-4a9e-9014-4722e134ddaa None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:48.440395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 146/294] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:48 2022] GET /identity/v3/role_assignments?user.id=b534e4e8ee27465bbfa8ba4626905c1a&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:48.588822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab1ec7-38fc-45d0-b642-2774df4877b6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.590521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab1ec7-38fc-45d0-b642-2774df4877b6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.590807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab1ec7-38fc-45d0-b642-2774df4877b6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.592468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 149/295] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:48.599101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.599685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.600180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.622016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:48.623628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:48.631327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa65-ca24-4d97-93ec-5f9913dcd8ef None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.631601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa65-ca24-4d97-93ec-5f9913dcd8ef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.631880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa65-ca24-4d97-93ec-5f9913dcd8ef None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.633664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 150/296] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:48.639206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:48.639499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:48.639776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:48.682676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:48.683921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:48.700130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 147/297] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:48.763256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 151/298] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:48.996994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.041822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.066875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.103480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521903ec-1922-4296-925c-a6e0327f5b29 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.133865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.141665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.142306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.142582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.142854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.143952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.145370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c772564-98bf-4a51-9764-685ca018b45a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.147104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 148/299] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:49.152756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e93eb8-f399-4f66-9f73-a9c6e5a00c0d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.153037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e93eb8-f399-4f66-9f73-a9c6e5a00c0d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.153131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb89ae5-b432-4d57-80a6-b9d2ba82b8a5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.153309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e93eb8-f399-4f66-9f73-a9c6e5a00c0d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.154689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 149/300] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:49.155230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e93eb8-f399-4f66-9f73-a9c6e5a00c0d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.196884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.197517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.197816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.198089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.208310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-28ba0d5f-bcff-4243-bbdc-52e909f8d620 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:49.212965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:49.213857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:49.214672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 152/301] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:49.215991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.216166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.216388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.235966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:49.237161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:49.272586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e93eb8-f399-4f66-9f73-a9c6e5a00c0d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.310578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.318127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 153/302] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:49.318266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.318266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.318266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] PATH_INFO: `/v3/groups` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.318512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.318592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.318592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.321641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.336422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 150/303] 10.222.0.82 () {60 vars in 1134 bytes} [Thu Aug 18 21:41:49 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:49.405880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.435071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a94fb-7a2a-4938-8354-7658288ecb3b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.472915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.473543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.473852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.474123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.474808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:49.475112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.476578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.486373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 154/304] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:49.492041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.602635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd602-ceaf-4155-b8d4-c1c1c68da93e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.625754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.645588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.646351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.646813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.647138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] PATH_INFO: `/v3/users/973652f0c17743b0a976a6a72bd795af` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.666518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] RBAC: Authorizing `identity:get_user(user_id=973652f0c17743b0a976a6a72bd795af)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.668017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.669694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 151/305] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/users/973652f0c17743b0a976a6a72bd795af => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:49.678075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.739696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be585ff5-9a00-4f30-b8ea-fd31c0b7c85b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.754499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.779883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.780606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.780944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.781283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.784017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:49.793908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:49.799243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:49.800413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:49.801385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:49.806042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 155/306] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:49.807722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:49.815038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8796-a358-4606-baa0-b7fa8a45faa1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.860010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.861020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.861444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.862008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.862888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, user.id=973652f0c17743b0a976a6a72bd795af, scope.system=all) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:49.863637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.865233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.881729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 152/307] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/role_assignments?user.id=973652f0c17743b0a976a6a72bd795af&scope.system=all&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:49.924938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213d01-a1d2-44af-9254-daa6d0f4b8de None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:49.965134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:49.965833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:49.966112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:49.966378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:49.967051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:49.967413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:49.968871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:49.982623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 156/308] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:49.985033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:50.092540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21114-c2e1-4dcc-b6e4-78b10437b2b9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:50.110427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:50.129344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:50.130011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:50.130329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:50.130610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] PATH_INFO: `/v3/users/a228897daf7a43b29be67d2cb12e3b11` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:50.131728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] RBAC: Authorizing `identity:get_user(user_id=a228897daf7a43b29be67d2cb12e3b11)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:50.133151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd454553-3e7f-49f5-a179-37d0acbb1680 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:50.134967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 153/309] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:49 2022] GET /identity/v3/users/a228897daf7a43b29be67d2cb12e3b11 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:50.141218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:50.204252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:50.249064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606883-5049-4a00-8c4d-0e5606797e14 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:50.283339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:50.284063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:50.284339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:50.284612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] PATH_INFO: `/v3/system/users/a228897daf7a43b29be67d2cb12e3b11/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:50.286188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a228897daf7a43b29be67d2cb12e3b11, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:50.287570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851da3b-efdf-42be-a0fe-2ca4be261ad6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:50.298876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 157/310] 10.222.0.82 () {58 vars in 1334 bytes} [Thu Aug 18 21:41:50 2022] PUT /identity/v3/system/users/a228897daf7a43b29be67d2cb12e3b11/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:41:50.477202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45b26-be53-43ea-b2ba-2277437d4d2b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:50.477585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45b26-be53-43ea-b2ba-2277437d4d2b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:50.478112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45b26-be53-43ea-b2ba-2277437d4d2b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:50.480673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 154/311] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:50.486882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302550-17da-48fb-887a-443402aa8dad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:50.487362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302550-17da-48fb-887a-443402aa8dad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:50.487826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302550-17da-48fb-887a-443402aa8dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:50.519585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e8302550-17da-48fb-887a-443402aa8dad None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:50.521550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e8302550-17da-48fb-887a-443402aa8dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:50.638663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 158/312] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:50.906115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a45b26-be53-43ea-b2ba-2277437d4d2b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.016136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a45b26-be53-43ea-b2ba-2277437d4d2b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:51.054279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:51.055263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.055785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.056261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.067604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:51.072488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:51.073366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:51.075012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:51.077023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 155/313] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:51.083366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8302550-17da-48fb-887a-443402aa8dad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.137852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:51.188508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8302550-17da-48fb-887a-443402aa8dad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:51.224778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:51.225402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.225759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.226037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.226710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:51.227015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:51.228397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:51.238056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 159/314] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:51.243592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.298886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:51.353535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775de67-58a6-45dd-8dda-1bdcffacc978 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:51.394204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:51.394850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.395130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.395407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] PATH_INFO: `/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.396568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] RBAC: Authorizing `identity:get_user(user_id=73c8de8efaad4ab9b8fc43a382a1e8e4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:51.398067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e647-85dc-4835-a0c5-2e8853a0914e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:51.399857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 156/315] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/users/73c8de8efaad4ab9b8fc43a382a1e8e4 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:51.405816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.497688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe297-b966-449b-9f7f-f2d3044ea7cf None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.498009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe297-b966-449b-9f7f-f2d3044ea7cf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.498198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe297-b966-449b-9f7f-f2d3044ea7cf None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.499618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 157/316] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:51.506086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.506761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.506761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.519110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24554-4119-4cf7-ab87-15c0824cfce7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:51.526421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:51.528563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:51.534390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:51.556411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:51.557023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.557299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.557637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.607930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 158/317] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:51.640862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:51.642378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:51.748495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 160/318] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:51.754065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.801230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:51.857993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eded7-26ae-4b99-8575-f31479f2701f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:51.876589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:51.893906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:51.894920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:51.895434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:51.895804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:51.896121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:51.897542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:51.899225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:51.901353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 159/319] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:51.908464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.001433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8727-e8fc-4ab5-b28e-ee3f6ff83e63 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.023899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac7398-7156-4fe3-91a3-332ef11bf83c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.038217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.038854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.039127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.039224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:52.039395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.040740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:52.042189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c60d24-bc5d-4979-a441-0a62a1c4f00d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:52.044401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 161/320] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:52.045504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1d0-6523-4166-bdf7-89b50ebc81bc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.045802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1d0-6523-4166-bdf7-89b50ebc81bc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.046071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1d0-6523-4166-bdf7-89b50ebc81bc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.047417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 162/321] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:52.051964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ac1d0-6523-4166-bdf7-89b50ebc81bc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.067890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.068505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.068784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.069061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.069811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, scope.project.id=2185553cecfb47fda5c928ec469ac6a3, user.id=73c8de8efaad4ab9b8fc43a382a1e8e4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:52.070123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:52.071550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e98675-30eb-4865-8645-4904a4c99ce6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:52.085633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 160/322] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:41:51 2022] GET /identity/v3/role_assignments?user.id=73c8de8efaad4ab9b8fc43a382a1e8e4&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:52.085633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d801-4304-423f-b3d9-d302877ffd4e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.085633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d801-4304-423f-b3d9-d302877ffd4e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.085633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d801-4304-423f-b3d9-d302877ffd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.098862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2a36d801-4304-423f-b3d9-d302877ffd4e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:52.100082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2a36d801-4304-423f-b3d9-d302877ffd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:52.169424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:52.193095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ac1d0-6523-4166-bdf7-89b50ebc81bc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.206044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 161/323] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:52.243801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.244421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.244701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.244975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] PATH_INFO: `/v3/groups` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.245681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:52.247159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7eae6-472a-488f-b285-45e9058f8c3e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:52.268405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 163/324] 10.222.0.82 () {60 vars in 1134 bytes} [Thu Aug 18 21:41:52 2022] POST /identity/v3/groups => generated 226 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:41:52.342889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88212-0aa9-401c-9e0f-73a0dc1f04a7 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.343208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88212-0aa9-401c-9e0f-73a0dc1f04a7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.343454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa88212-0aa9-401c-9e0f-73a0dc1f04a7 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.344856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 162/325] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:52.350861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.351190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.351415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.372128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:52.373366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:52.450109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 164/326] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:52.477196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa88212-0aa9-401c-9e0f-73a0dc1f04a7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.590431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa88212-0aa9-401c-9e0f-73a0dc1f04a7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.600757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:52.633241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.633902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.634182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.634455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.648641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:52.653888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:52.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:52.655666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 163/327] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:52.661887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.729068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.755679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:52.793990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aec9d5-5167-4b9c-addf-2c70c95cec1d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.835583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.836209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.836488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.836763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.837443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:52.837907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:52.839848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:52.853772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 165/328] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:52.856342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.873531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcf278-957d-4510-b488-c36c3ee22d4b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:52.922237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:52.923000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:52.923349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:52.923692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:52.935733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:52.941364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:52.942261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:52.943050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 164/329] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:52.948643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:52.989939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:52.991971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526365-8774-4abf-a9d3-97c2471b05ea None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:53.044557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:53.045200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:53.045482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:53.045792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] PATH_INFO: `/v3/users/973652f0c17743b0a976a6a72bd795af` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:53.047222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] RBAC: Authorizing `identity:get_user(user_id=973652f0c17743b0a976a6a72bd795af)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:53.049741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:53.052136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 166/330] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:52 2022] GET /identity/v3/users/973652f0c17743b0a976a6a72bd795af => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:53.058934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:53.063745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:53.094008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae751172-d3ce-422b-8921-d1470f333ed8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:53.158399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:53.159377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:53.159854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:53.160319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:53.161315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:53.161848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:53.163509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:53.174629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 165/331] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:53.180576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:53.204962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dfcad-db69-454c-85dd-c8ec8cc959a4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:53.248453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:53.249113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:53.249390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:53.249702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] PATH_INFO: `/v3/system/users/973652f0c17743b0a976a6a72bd795af/roles/4f33402556ca4c3d8dcafd31f341a974` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:53.251131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=973652f0c17743b0a976a6a72bd795af, role_id=4f33402556ca4c3d8dcafd31f341a974)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:53.252947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:53.265167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 167/332] 10.222.0.82 () {58 vars in 1334 bytes} [Thu Aug 18 21:41:53 2022] PUT /identity/v3/system/users/973652f0c17743b0a976a6a72bd795af/roles/4f33402556ca4c3d8dcafd31f341a974 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:41:53.318485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:53.332734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34120c-1831-44a3-89ea-528654dd1ba6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:53.373287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:53.374023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:53.374305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:53.374582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] PATH_INFO: `/v3/users/a228897daf7a43b29be67d2cb12e3b11` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:53.375886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] RBAC: Authorizing `identity:get_user(user_id=a228897daf7a43b29be67d2cb12e3b11)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:53.377482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6ea90-d02b-4fae-8958-a8ca75556980 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:53.380013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 166/333] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:53 2022] GET /identity/v3/users/a228897daf7a43b29be67d2cb12e3b11 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:53.385871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:53.469298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:53.514849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217e928-33cf-46d8-91e5-6ee6bfa02c89 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:53.563603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:53.564239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:53.564522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:53.564857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:53.565926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] RBAC: Adding query filter params (role.id=4f258cbfb9214c3f8093f3eca5da61b9, user.id=a228897daf7a43b29be67d2cb12e3b11, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:53.566243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:53.567762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032eaea-4bf2-457e-ad94-861762560a8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:53.588267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 168/334] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 21:41:53 2022] GET /identity/v3/role_assignments?user.id=a228897daf7a43b29be67d2cb12e3b11&scope.system=all&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:54.749530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6505-4e02-49e7-8b02-cf29f73f9e12 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:54.750113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6505-4e02-49e7-8b02-cf29f73f9e12 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:54.750113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e6505-4e02-49e7-8b02-cf29f73f9e12 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:54.751576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 167/335] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:54.757974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83bf1-c1ad-4ac7-8c4e-73d187ddc5d7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:54.758392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83bf1-c1ad-4ac7-8c4e-73d187ddc5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:54.758580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83bf1-c1ad-4ac7-8c4e-73d187ddc5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:54.782559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a4e83bf1-c1ad-4ac7-8c4e-73d187ddc5d7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:54.783761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a4e83bf1-c1ad-4ac7-8c4e-73d187ddc5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:54.867642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 169/336] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:55.138316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae52bb5-72bb-480c-85e5-f738ee7ca66b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.138702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae52bb5-72bb-480c-85e5-f738ee7ca66b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.138884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae52bb5-72bb-480c-85e5-f738ee7ca66b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.140485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 168/337] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:55.145935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef898-1781-4090-8db1-93a71535a7ca None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.146224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef898-1781-4090-8db1-93a71535a7ca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.146504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef898-1781-4090-8db1-93a71535a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.170173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-264ef898-1781-4090-8db1-93a71535a7ca None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:55.171428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-264ef898-1781-4090-8db1-93a71535a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:55.234569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:55.258739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 170/338] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:55.396507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae52bb5-72bb-480c-85e5-f738ee7ca66b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:55.486139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26311-2243-4288-9a61-3a1cb4f0162f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.486456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26311-2243-4288-9a61-3a1cb4f0162f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.486684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26311-2243-4288-9a61-3a1cb4f0162f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.488288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 171/339] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:55.493578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.493921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.494192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.523227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:55.524421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:55.530570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae52bb5-72bb-480c-85e5-f738ee7ca66b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:55.578622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:55.579698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.580190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.581103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.599143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:55.604018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:55.604931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:55.606354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 169/340] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:55.614835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:55.658028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:55.669630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 172/341] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:41:55.761793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c78f4-066e-4f84-b932-bfae523a6bca None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:55.801123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:55.801786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:55.802079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:55.802358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:55.803040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:55.803348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:55.804750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:55.814376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 170/342] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:55.819724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:55.929963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:55.941532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7d48-1410-41c7-9412-fab025555464 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:55.974236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.029481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.030187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.030472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.030747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.093996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.144731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:56.146268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:56.155544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2c46-29eb-4e45-bb67-40c59aa93de6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.219968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.220607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.221236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.221796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.246332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:56.252057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:56.253141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:56.254146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:56.261616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 171/343] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:41:56.262910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.268780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 173/344] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:56.269833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.287393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.389163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf36c6-d160-4214-becb-d3ac6090aba5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.431109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.432377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.432807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.433532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.434386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:56.434838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:56.436466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:56.445994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 174/345] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:56.446986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.489221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f8c6f-b9e3-4d0c-92ba-31716438d691 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.537275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.564345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.565154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.565558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.566044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.566590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb283ff-ade6-42df-934d-6ed26af7d649 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.573521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.580067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:56.585959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:56.587066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:41:56.588148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:56.589782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 172/346] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:56.590730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.615753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.616929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.617355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.617822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.619135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:56.622045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4caf95-6ad3-460e-b837-7121e26fe687 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:56.624028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 175/347] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:56.626040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb77f2-e3ad-4116-9a87-888b2f69f85c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.626497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb77f2-e3ad-4116-9a87-888b2f69f85c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.626916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb77f2-e3ad-4116-9a87-888b2f69f85c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.629337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 176/348] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:56.630375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb77f2-e3ad-4116-9a87-888b2f69f85c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.726538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a492-441b-4144-9758-4154dbaf219a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.759224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb77f2-e3ad-4116-9a87-888b2f69f85c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:56.776292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.777263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.777263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.778157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.805822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:56.806822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:56.807209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:56.807489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:56.809954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:56.809954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:56.810623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:56.820215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 177/349] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:56.821840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:56.833365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.865158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:56.877001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:56.879553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892c574-2c92-49f1-8625-29bd5d241bb0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:56.966033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6d8a3-15b7-491b-94e1-5ef0947223c3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.008356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 173/350] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:57.010601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.010788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.010902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.012362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.012468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.012468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.012468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.020517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.022121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1959d1-2167-460f-adee-389c38b93132 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.024212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 178/351] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:41:56 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:57.025378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04117-d567-4ba3-8e35-09d8d40f0f5a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.025684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04117-d567-4ba3-8e35-09d8d40f0f5a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.025953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04117-d567-4ba3-8e35-09d8d40f0f5a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.028200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 179/352] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:57.028762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04117-d567-4ba3-8e35-09d8d40f0f5a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.030951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:57.032892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:57.108967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 174/353] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:57.109974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.117768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.144378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.159259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04117-d567-4ba3-8e35-09d8d40f0f5a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.202879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.203521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.203765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.204156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.236198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72fa36-d70d-42d9-946f-bce569ec0b88 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.272584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.273443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.273897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.274384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.275682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.278198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.279982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 175/354] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:57.280870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.286699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.288173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992720-9435-4b8b-b21e-85d280862637 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.374854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.409343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.416142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22b21-cbe8-4b0f-abd3-98742eed22d4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.417263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 180/355] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:57.419070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.419387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.419670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.440208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:57.441519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:57.461533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.462440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.462725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.462996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.464291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=995426667c154ea2a375fda95b1f26ff) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:57.464291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.465485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37121-41d9-41c3-889d-ca03be7c474b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.472875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 176/356] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:57.474383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bee5-4dbc-4b91-b1d9-d5170c794ff1 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.474748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bee5-4dbc-4b91-b1d9-d5170c794ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.475095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bee5-4dbc-4b91-b1d9-d5170c794ff1 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.476858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 177/357] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:41:57.482681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694bee5-4dbc-4b91-b1d9-d5170c794ff1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.532161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 181/358] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:57.534497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.610300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694bee5-4dbc-4b91-b1d9-d5170c794ff1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.657159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.658079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.658496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.658897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.660615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.662488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.663208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.666701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 178/359] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:57.666848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58990a-99f6-4cd3-b8ef-73fbf83e1262 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.666928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.721166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.722207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.722618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.723027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.729274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.741822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e70e8b3a-0f3f-46f5-b2ff-164a29c30b54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:57.747402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:57.748274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:41:57.749058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:57.753791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 182/360] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:57.755170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.755985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.756276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.779573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:57.781005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:57.799656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff056d-41a1-46c7-ae75-cbf09ae233f1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.837459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:57.838133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:57.838408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:57.838731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:57.840436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:57.841962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:57.843787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 179/361] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:57.844538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.859267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 183/362] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:57.860333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:57.954497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.960374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303d5c4-9f86-4932-90a6-94b1f30ab810 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.963141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:57.974199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20f366-96de-429c-9b54-b93ed9b3f55f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:57.999557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.000234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.000520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.000791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.001524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=995426667c154ea2a375fda95b1f26ff) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:58.001862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.003816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.010812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 180/363] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.011739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.012715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.013407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.013733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.014016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.014704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:58.015014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.016427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.029740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 184/364] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:41:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.030778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.148562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a0d1-43bc-459a-b40a-aa6bae2918af None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.178095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6dd7b-b4d1-4ca7-a812-0392224dbd5e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.191344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.192005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.192413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.192686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.208896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:58.216950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:58.217922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:41:58.218717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 185/365] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:58.219384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.227901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.228548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.228824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.229127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.230985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.232669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.234680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 181/366] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.235297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.236671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.271180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.338800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304055e-f83e-4d37-a91c-6e1dd4c18737 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.354309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758df7-0940-4f13-88ee-1432863eb15e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.375393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.376352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.376855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.377331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.392317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.393075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.393358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.393652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.406740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:41:58.411648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:41:58.412629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:41:58.413427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 182/367] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:41:58.414273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.461358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.463807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.480914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.559139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.577611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28a87f-0e89-442e-be57-eafe9308ab00 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.600465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 186/368] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:58.601397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.617138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.618264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.618739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.619200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.620055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:58.621082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.622694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.633095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 183/369] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.634246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.718509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d526-aeae-4b2d-8139-b9a19af079c3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.736379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.762764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.763377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.763666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.763933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.764672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=2185553cecfb47fda5c928ec469ac6a3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:58.764985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.766453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.769315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d31a-17da-485a-91f3-3fe6c174ae3a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.772578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 187/370] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.773256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.832705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.834077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.834560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.835027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.835941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:58.836450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.838096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.848299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 184/371] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.849377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.871345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:58.892836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df5c6-5521-447b-84c5-b52986e6c6c8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.931726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:58.932384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:58.932672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:58.932912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:58.934224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:58.935802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:58.940569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 188/372] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:58.943543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:58.974875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47984c3a-1568-487c-864e-9e9d9ea944db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:58.997667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.017280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.017785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.018077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.018356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] PATH_INFO: `/v3/users/973652f0c17743b0a976a6a72bd795af` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.019952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] RBAC: Authorizing `identity:get_user(user_id=973652f0c17743b0a976a6a72bd795af)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.021699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.027185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 185/373] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/users/973652f0c17743b0a976a6a72bd795af => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:59.027185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:59.088004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754bb1-c560-4841-8c42-ad686a7c03f2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.134746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.135521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.135906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.136302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.137089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.170721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f25657-9e4c-45e0-bfbb-8ded2bc13250 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.211953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.212834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.213404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.213981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.215536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.217436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.219621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 186/374] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:59.221137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:59.258531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.260140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.341254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04fab-0cca-41ce-bf83-455075836bbf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.361570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.383032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.383892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.384471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.384934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.386728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] RBAC: Adding query filter params (role.id=4f33402556ca4c3d8dcafd31f341a974, user.id=973652f0c17743b0a976a6a72bd795af, scope.system=all) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:59.387356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.389900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.404712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 187/375] 10.222.0.82 () {56 vars in 1308 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/role_assignments?user.id=973652f0c17743b0a976a6a72bd795af&scope.system=all&role.id=4f33402556ca4c3d8dcafd31f341a974 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:59.406031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:59.421763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.495132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 189/376] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:41:59.502075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:59.564935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1b695-f58e-45ef-976d-4f51eea32ed9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.603187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.604062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.604491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.604983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.606405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=2185553cecfb47fda5c928ec469ac6a3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:41:59.606901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.608512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aecd9-7899-41f3-be69-a424a48de8be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.614986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 188/377] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:59.678190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d2667-d740-463e-b9a4-0fb8d0d01d41 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.725259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.725948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.726225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.726494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] PATH_INFO: `/v3/projects/940373c3ec634186892a510e45b0afa3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.730991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] RBAC: Authorizing `identity:get_project(project_id=940373c3ec634186892a510e45b0afa3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.732482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.734604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 190/378] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/projects/940373c3ec634186892a510e45b0afa3 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:41:59.734852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04440e-0d97-4daa-bc2d-83bdc82303c1 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.735157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04440e-0d97-4daa-bc2d-83bdc82303c1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.735508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04440e-0d97-4daa-bc2d-83bdc82303c1 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.737093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 189/379] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:41:59.742185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:41:59.742649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.742957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.743217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.778027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.779828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:41:59.781083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:41:59.867376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff322-f714-44cc-8c85-e7f93a187225 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:41:59.877536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:41:59.886662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 190/380] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:41:59.905383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:41:59.906683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:41:59.907154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:41:59.907616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] PATH_INFO: `/v3/groups/a7970de1471547668b3236212df5a473` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:41:59.913102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] RBAC: Authorizing `identity:get_group(group_id=a7970de1471547668b3236212df5a473)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:41:59.914864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:41:59.917029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 191/381] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/groups/a7970de1471547668b3236212df5a473 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:41:59.923402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:00.062224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f40c-bb95-4b36-bb4e-380f07ccfefd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:00.117827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:00.118370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.118648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.118922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.119659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] RBAC: Adding query filter params (group.id=a7970de1471547668b3236212df5a473, role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=940373c3ec634186892a510e45b0afa3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:00.119967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:00.121435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:00.130157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 191/382] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:41:59 2022] GET /identity/v3/role_assignments?group.id=a7970de1471547668b3236212df5a473&scope.project.id=940373c3ec634186892a510e45b0afa3&role.id=a8f69336710f474db0973fac38e02b80 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:00.314840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:00.403092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:00.432277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb663-05cb-43cf-9760-25a1e482b30f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:00.471081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:00.471784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.472063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.472335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.485940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:00.491260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:00.492112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:00.492904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 192/383] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:00.499149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:00.582087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:00.631657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb05b5-6553-4acd-b886-442f1f5657e8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:00.676441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:00.676980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.677263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.677536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.678242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:00.678542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:00.680118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7c16d-825d-400a-ae88-292e114d241d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:00.694395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 192/384] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:00.700672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:00.825237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731e2a-4d80-449e-885c-6ef28a1089ee None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.825570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731e2a-4d80-449e-885c-6ef28a1089ee None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.825844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731e2a-4d80-449e-885c-6ef28a1089ee None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.827185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 193/385] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:00.835541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.835822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.836103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.856884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccd0d-502d-4fa6-8604-261af5a0b806 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:00.869294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:00.871086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:00.922390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:00.923060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:00.923287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:00.923555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:00.925347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:00.926896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:00.929838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 193/386] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:00 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:00.939766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:00.946690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:01.006269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 194/387] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:01.188704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582c96e-c5bb-4950-80be-d9774aaf31ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:01.226126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:01.226765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.227041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.227311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.228447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:01.230509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:01.232148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 194/388] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:00 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:01.238004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:01.316240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:01.354278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6ae1-2535-4d98-8c0f-84ed394bb798 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:01.399471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:01.400378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.400795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.401206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/groups/203190ddb7884ee9a3d34185741c9c58/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.404231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, group_id=203190ddb7884ee9a3d34185741c9c58, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:01.406274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f35e31-3e2d-41d8-b58b-997e48985951 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:01.428293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 195/389] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 21:42:01 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/groups/203190ddb7884ee9a3d34185741c9c58/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:01.604167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:01.698684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:01.702929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4764b5c-56f3-4606-9a9d-4c3718ad71a5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.703498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4764b5c-56f3-4606-9a9d-4c3718ad71a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.703972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4764b5c-56f3-4606-9a9d-4c3718ad71a5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.706899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 196/390] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:01.719922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.720442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.721043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.740478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67499b-489c-406c-a607-01d31de1c28f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:01.747414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:01.748804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:01.775470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:01.776351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.777313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.778011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.791612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:01.804317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:01.805406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:01.806630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:01.809573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 195/391] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:01.819134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:01.862404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 197/392] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:01.936953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b37c7-00bc-4bfa-9154-e4f7d84f19e8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:01.969224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:01.975329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:01.975954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:01.976234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:01.976508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:01.977182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:01.977533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:01.978955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:01.987935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 196/393] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:01.993671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.051586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:02.122340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694ba33-b32e-4e51-8d37-cc85e66bec0d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:02.165698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:02.166344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.166624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.166898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.168548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:02.170082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:02.171651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 198/394] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:01 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:02.177202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.286776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910f32-4760-4581-b24e-35fd459a2f11 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:02.322218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:02.322862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.323141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.323414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.325113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:02.326612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:02.328641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 197/395] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:02 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:02.334442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.403029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.485153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c1b0-1e75-4bb6-a352-4818cb412bcb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:02.497763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:02.508277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:02.537676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:02.538366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.538890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.539168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff/groups/203190ddb7884ee9a3d34185741c9c58/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.542901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=995426667c154ea2a375fda95b1f26ff, group_id=203190ddb7884ee9a3d34185741c9c58, role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:02.544971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58f804-2c67-499f-b296-e893bedcc2c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:02.549327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7275dd7-203e-4fe7-a2e5-8717f15bdb1f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:02.562305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 199/396] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 21:42:02 2022] PUT /identity/v3/projects/995426667c154ea2a375fda95b1f26ff/groups/203190ddb7884ee9a3d34185741c9c58/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:02.566420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddfcc-6f3b-464b-9e97-251b6c4cfa7d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.566749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddfcc-6f3b-464b-9e97-251b6c4cfa7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.567124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ddfcc-6f3b-464b-9e97-251b6c4cfa7d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.568665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 200/397] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:02.576472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.576779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.577013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.604516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:02.605810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:02.626396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:02.627069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.627352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.627631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.662794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:02.668643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:02.669550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:02.670367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:02.678278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 198/398] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:02.687786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.704432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 201/399] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:02.816779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8653-9d74-49f1-ba60-a9ea7858104c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:02.840213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:02.859714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:02.860340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.860618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.860941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.861649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:02.861949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:02.863416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e90c08-e850-47dd-94a2-802e370dcde8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:02.876505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 199/400] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:42:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:02.882081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:02.986682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5529c-5cde-4feb-a85b-20c2fe3b4229 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.987158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5529c-5cde-4feb-a85b-20c2fe3b4229 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.987158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5529c-5cde-4feb-a85b-20c2fe3b4229 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:02.989173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 200/401] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:02.994658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:02.995201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:02.995730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.017165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:03.019519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:03.048074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc66c0-aca4-4f87-92cb-65fbec63663c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.059603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.097648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.098640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.099540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.100104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.102085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.104508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.107700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 202/402] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:02 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:03.108939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.115151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 201/403] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:03.117045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.255183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883c4c-259a-4f1c-a653-ddc659cea912 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.299883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa8da6-4e9d-4e62-8957-bf708267fddb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.325674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.333530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.334196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.334478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.334820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.359064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:03.363856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:03.364807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:03.365617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:03.366310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 203/404] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:03.373048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.383948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.384600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.384883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.385216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.390292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.391756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.393668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 202/405] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:03.401073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.469637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.510650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75c74-65c0-4a9e-bd08-c398f1c5c727 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.535279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f29c2-2397-448f-8ec1-541ea7a1c9f9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.559748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.560928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.561680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.562706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.563904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:03.564696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.567032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.572921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.574084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.574363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.574630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/groups/203190ddb7884ee9a3d34185741c9c58/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.576612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, group_id=203190ddb7884ee9a3d34185741c9c58, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.578081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.579241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 204/406] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:03.580647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.594878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 203/407] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 21:42:03 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/groups/203190ddb7884ee9a3d34185741c9c58/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:03.595865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.647827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.724865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3c8a-edfe-4beb-9859-2ccc4d1d5a01 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.735274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.751055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2b84-dd1e-4a34-8cd0-435ab6385505 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.769929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.771060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.772105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.772769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.774495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.776631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.780336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 204/408] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:03.785443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.796678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.797406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.798187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.798187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.815063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:03.820926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:03.821899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:03.822730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 205/409] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:03.828638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:03.910217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750eddc-9f87-4c2d-b766-2e6c9bdc29b3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:03.937984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:03.989415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:03.990101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:03.990361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:03.990638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:03.991786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:03.993257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:03.999409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4509c-8f79-45dd-bd2b-66f77d9e93aa None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.002001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 205/410] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:04.003313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.041708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.042526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.043756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.044042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.044721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:04.045077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:04.046528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058898-8911-4748-912d-3afe097c4a62 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:04.057569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 206/411] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:04.059586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7cdb8-b91a-4b50-b12d-a6b46852a08b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.059868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7cdb8-b91a-4b50-b12d-a6b46852a08b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.060401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7cdb8-b91a-4b50-b12d-a6b46852a08b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.061788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 207/412] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:04.063099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7cdb8-b91a-4b50-b12d-a6b46852a08b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.084656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:04.135404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eb8eb-9fa2-494d-9ecf-a8424f038779 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.178515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7cdb8-b91a-4b50-b12d-a6b46852a08b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.182735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.183754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.184228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.184704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3/groups/203190ddb7884ee9a3d34185741c9c58/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.187735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] RBAC: Authorizing `identity:create_grant(project_id=2185553cecfb47fda5c928ec469ac6a3, group_id=203190ddb7884ee9a3d34185741c9c58, role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:04.190331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a259540-adff-41e7-871d-e41b1fbb849f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:04.211122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 206/413] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 21:42:04 2022] PUT /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3/groups/203190ddb7884ee9a3d34185741c9c58/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:04.213446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.213855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.214886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.218296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.219245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.219726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.220153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] PATH_INFO: `/v3/groups/a7970de1471547668b3236212df5a473` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.221586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] RBAC: Authorizing `identity:get_group(group_id=a7970de1471547668b3236212df5a473)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:04.223554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:04.225838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 208/414] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:04 2022] GET /identity/v3/groups/a7970de1471547668b3236212df5a473 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:04.235505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:04.236828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:04.237169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.279830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:04.327365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 207/415] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:04.411770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:04.437087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944ffd-3fde-4e9e-8f12-fff0e87ca6f8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.477419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155576a6-6928-44e1-984c-303244570a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.478492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155576a6-6928-44e1-984c-303244570a9e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.479000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155576a6-6928-44e1-984c-303244570a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.479496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155576a6-6928-44e1-984c-303244570a9e None admin] PATH_INFO: `/v3/projects/940373c3ec634186892a510e45b0afa3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.481759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155576a6-6928-44e1-984c-303244570a9e None admin] RBAC: Authorizing `identity:get_project(project_id=940373c3ec634186892a510e45b0afa3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:04.483504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155576a6-6928-44e1-984c-303244570a9e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:04.485800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 209/416] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:04 2022] GET /identity/v3/projects/940373c3ec634186892a510e45b0afa3 => generated 344 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:04.496683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.605651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ba000-cfd5-4002-8e23-ab4f53600f96 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.640919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.641588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.641899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.642174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] PATH_INFO: `/v3/projects/940373c3ec634186892a510e45b0afa3/groups/a7970de1471547668b3236212df5a473/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.644103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=940373c3ec634186892a510e45b0afa3, group_id=a7970de1471547668b3236212df5a473, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:04.645544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e08c8-ac52-4f9b-bdf0-7e15647cafb2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:04.657052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155576a6-6928-44e1-984c-303244570a9e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.660484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 208/417] 10.222.0.82 () {58 vars in 1442 bytes} [Thu Aug 18 21:42:04 2022] PUT /identity/v3/projects/940373c3ec634186892a510e45b0afa3/groups/a7970de1471547668b3236212df5a473/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:04.827757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5991a7-163c-46ce-889f-c049c867c4be None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.828406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5991a7-163c-46ce-889f-c049c867c4be None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.828650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155576a6-6928-44e1-984c-303244570a9e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:04.829117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5991a7-163c-46ce-889f-c049c867c4be None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.832060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 209/418] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:04.836658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.837231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.837742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.871981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:04.873489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:04.874829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:04.875474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:04.875758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:04.876036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:04.890152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:04.897151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:04.898085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:04.899462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 210/419] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:04.904749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:04.921282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:04.923608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:04.986743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 210/420] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:05.059735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ebc5-78cc-4091-9aba-0c326b6473aa None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:05.120807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:05.121455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.121726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.122058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.122673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:05.122981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:05.124359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:05.133057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 211/421] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:05.138860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.314317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa416a-79ec-4a33-8cb0-4091ad5051b9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:05.330727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.366010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:05.369816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.370256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.370709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.426046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:05.437631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:05.463142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976115b-426f-4543-9132-eafa46dc3780 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:05.505218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:05.507130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:05.517160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:05.517836 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.518118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.518389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.537618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:05.547325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:05.548270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:05.549063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:05.554436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 212/422] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:05.559562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.733042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 211/423] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:05.739080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.742009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a537770-70d7-4343-ae1e-2b28c96e0b1b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:05.785799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:05.786774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.787266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.787833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.788707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:05.789204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:05.790802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:05.793629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:05.805392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 213/424] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:05.816164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.873966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:05.889953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7ee1a-dab4-4d80-a97a-73cd4ac08559 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:05.928575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:05.929313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.929618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.929892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.931094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:05.932545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483548a7-4c37-4aef-80ab-a7c9b934884f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:05.934449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 212/425] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:05.935743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7d450-16e3-488e-b535-9610d634b003 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:05.936023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7d450-16e3-488e-b535-9610d634b003 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:05.936351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7d450-16e3-488e-b535-9610d634b003 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:05.941663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 213/426] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:05.942553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7d450-16e3-488e-b535-9610d634b003 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:05.978153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af88-e510-44a3-95c7-7003f66f5b18 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.016244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.017492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.018154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.018904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.053208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7d450-16e3-488e-b535-9610d634b003 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.086047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.092977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.093844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.094265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.094698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.096433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.098369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11e60d-610a-49d6-a282-aa848c60e039 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.106082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.106615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 214/427] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:06.108500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.108742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.109321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.109825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.110449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.129126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:06.130717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:06.229035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 215/428] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:06.229035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.240556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 214/429] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:06.246518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.339011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efc95c-d870-4638-83d8-f4e715d68815 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.367673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.381850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.382785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.383218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.383631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.384523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=995426667c154ea2a375fda95b1f26ff) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:06.384977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.386918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.394083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 216/430] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:06.444150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.451388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43435-144a-4cac-a810-f4a1773d1f71 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.503290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.503869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.504150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.504427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] PATH_INFO: `/v3/projects/995426667c154ea2a375fda95b1f26ff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.505571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] RBAC: Authorizing `identity:get_project(project_id=995426667c154ea2a375fda95b1f26ff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.507114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.509054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 215/431] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/projects/995426667c154ea2a375fda95b1f26ff => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:06.515002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.620357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.679510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1ddc-de2b-4d64-96d5-eef658c3fc96 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.732253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.733279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.733754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.734482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.736061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.736907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb3d1b-0709-4460-9f87-4f792bbdccb4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.737762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a70ac9-d042-4fd8-9572-a6efb45f3031 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.739713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 217/432] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:06.742620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f80978-d7a2-4373-8b3d-f814a18941a3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.743086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f80978-d7a2-4373-8b3d-f814a18941a3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.743509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f80978-d7a2-4373-8b3d-f814a18941a3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.745191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 218/433] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:06.750592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f80978-d7a2-4373-8b3d-f814a18941a3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.784263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.785325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.785627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.785896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.799605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-caf14b32-2d4e-4a39-a87f-c7f2aabb80a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:06.804598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:06.805482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:06.806324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 216/434] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:06.812525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.812996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.813442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.829817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.833446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:06.834674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:06.860890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:06.886697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f80978-d7a2-4373-8b3d-f814a18941a3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:06.930547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 217/435] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:06.932178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:06.937365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:06.938076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:06.938417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:06.938775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:06.939571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=995426667c154ea2a375fda95b1f26ff) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:06.939943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:06.941454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbf3b4-5387-4e18-9f7d-95ab291ed831 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:06.949560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 219/436] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=995426667c154ea2a375fda95b1f26ff&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:07.034358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd99a65-2dec-44c2-bcf8-881188e87c93 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.034544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd99a65-2dec-44c2-bcf8-881188e87c93 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.034933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd99a65-2dec-44c2-bcf8-881188e87c93 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.036987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 220/437] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:07.046491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.047158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.047635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.058215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b316f1-984d-48c0-acb7-eb02d9e719b3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.067437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:07.069028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:07.099771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.100674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.101300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.101923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.103131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:07.103791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.106802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.127556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 218/438] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:42:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:07.129067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.155295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.159422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 221/439] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:07.225113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.253712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396094-ea56-4808-82bb-0258e74a071a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.296770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.303017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.303840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.304120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.304432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.363573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acbdca-a44e-4725-a40e-8585780e6b3d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.394019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.395482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.409112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.410805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.410805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.410805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.415489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.423388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:07.428141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:07.429013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:07.429948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 222/440] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:07.437569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.510847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 219/441] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 21:42:07.512196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.543851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4733b-6223-4e76-ad9f-3dd5ffd95685 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.554487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.578684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.579299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.579589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.579863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.580531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:07.580837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.582282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.590331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 223/442] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:07.591362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.616638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22211890-0e37-44ba-94b2-2f0f39199813 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.633284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.651429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.652056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.652339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.652866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.662696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:07.667319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:07.668208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:07.668990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:07.673288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 220/443] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:07.673990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.688849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00a85-6489-43aa-9264-954f4c630e23 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.722766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.723399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.723680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.723947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.725062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.726570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.732846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 224/444] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:07.733581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.774304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12b606-a1be-44a9-bf59-c34994d2d9c4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.778362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.809255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.809813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.810099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.810371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.835929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2559-fa4a-425b-ac53-d088f0e8f987 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.856621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:07.868925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:07.869534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:07.869834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:07.870104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:07.871354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:07.871614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.873035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.879262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:07.880724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:07.880837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 225/445] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:07.881656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.981768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 221/446] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:07.982615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:07.984321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bff4a-3b19-43ea-b128-9161536ed4fc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:07.985876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.018938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.019568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.019843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.020114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.021226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.022691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.024241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 226/447] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.024993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.060484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.082991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531092b5-ebba-4b95-ae25-8acd9c4ba58e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.117862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.118476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.118757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.119054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.123750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d282b-5365-4237-bff6-2c114c329bab None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.158187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.158818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.159094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.159366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] PATH_INFO: `/v3/projects/2185553cecfb47fda5c928ec469ac6a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.160495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] RBAC: Authorizing `identity:get_project(project_id=2185553cecfb47fda5c928ec469ac6a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.161962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.163556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 227/448] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/projects/2185553cecfb47fda5c928ec469ac6a3 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.164314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.191049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.192514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.204168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.266019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4e70-1515-440c-954a-f0824b195900 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.270226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.293267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 222/449] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:08.294132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.301313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.301955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.302232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.302504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.303228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=4f258cbfb9214c3f8093f3eca5da61b9, scope.project.id=2185553cecfb47fda5c928ec469ac6a3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:08.303535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.304961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.311346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 228/450] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=4f258cbfb9214c3f8093f3eca5da61b9 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.312070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.409693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087fc0b-43a1-4e27-b3bc-d128da71c300 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.432522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee8a4d-764c-4b5e-817e-b777fb224592 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.434573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.452211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.452855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.453138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.453432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] PATH_INFO: `/v3/groups/203190ddb7884ee9a3d34185741c9c58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.454914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] RBAC: Authorizing `identity:get_group(group_id=203190ddb7884ee9a3d34185741c9c58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.456714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.457695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 223/451] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/groups/203190ddb7884ee9a3d34185741c9c58 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.463383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.478476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.478476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.478476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.478822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] PATH_INFO: `/v3/projects/940373c3ec634186892a510e45b0afa3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.480573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] RBAC: Authorizing `identity:get_project(project_id=940373c3ec634186892a510e45b0afa3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.482105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.484025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 229/452] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/projects/940373c3ec634186892a510e45b0afa3 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.489983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.527099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.595957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770610c-f18e-4ca9-affc-95c4e8eb1b51 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.617097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832424-f140-40ca-a7fd-199040fd0096 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.639456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.640394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.640394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.640635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.641346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] RBAC: Adding query filter params (group.id=203190ddb7884ee9a3d34185741c9c58, role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=2185553cecfb47fda5c928ec469ac6a3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:08.641681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.643148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.649501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 224/453] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/role_assignments?group.id=203190ddb7884ee9a3d34185741c9c58&scope.project.id=2185553cecfb47fda5c928ec469ac6a3&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:08.661036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.661735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.662020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.662298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] PATH_INFO: `/v3/groups/a7970de1471547668b3236212df5a473` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.663535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] RBAC: Authorizing `identity:get_group(group_id=a7970de1471547668b3236212df5a473)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.665039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb35bd-4b8b-4c8a-8cd5-afb6cf255d28 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.667072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 230/454] 10.222.0.82 () {56 vars in 1180 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/groups/a7970de1471547668b3236212df5a473 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:08.673686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:08.784199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e57c3-3723-4ba9-a1fe-62c996ef4501 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:08.793627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:08.821498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:08.822155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:08.822440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:08.822716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:08.824008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] RBAC: Adding query filter params (group.id=a7970de1471547668b3236212df5a473, role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=940373c3ec634186892a510e45b0afa3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:08.824324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:08.825789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ad3d-b8c6-460c-a634-d2fe641cb12a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:08.831812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 225/455] 10.222.0.82 () {56 vars in 1376 bytes} [Thu Aug 18 21:42:08 2022] GET /identity/v3/role_assignments?group.id=a7970de1471547668b3236212df5a473&scope.project.id=940373c3ec634186892a510e45b0afa3&role.id=a8f69336710f474db0973fac38e02b80 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:11.098003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d160-005e-44f8-b51f-7d576eda0da9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.098545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d160-005e-44f8-b51f-7d576eda0da9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.098545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622d160-005e-44f8-b51f-7d576eda0da9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.099960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 231/456] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:11.106908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8610436-1ad9-4c26-a2b6-addbe99f119a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.107350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8610436-1ad9-4c26-a2b6-addbe99f119a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.107444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8610436-1ad9-4c26-a2b6-addbe99f119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.130761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e8610436-1ad9-4c26-a2b6-addbe99f119a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:11.131956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8610436-1ad9-4c26-a2b6-addbe99f119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:11.223018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 226/457] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:11.409577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5fe-6a48-44bf-b16c-18abaeea24c0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.410168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5fe-6a48-44bf-b16c-18abaeea24c0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.410623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5fe-6a48-44bf-b16c-18abaeea24c0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.412350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 232/458] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:11.418440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ea052-5781-443e-8a87-e88f534dd637 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.418978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ea052-5781-443e-8a87-e88f534dd637 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.419471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ea052-5781-443e-8a87-e88f534dd637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.441391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-235ea052-5781-443e-8a87-e88f534dd637 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:11.446668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-235ea052-5781-443e-8a87-e88f534dd637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:11.528752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 227/459] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:11.591161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de5fe-6a48-44bf-b16c-18abaeea24c0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:11.650583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:11.670189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372957c-2ba1-41ff-a96f-086472f06d37 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.670467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372957c-2ba1-41ff-a96f-086472f06d37 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.670667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372957c-2ba1-41ff-a96f-086472f06d37 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.672168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 228/460] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:11.677937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.678571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.679054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.721621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:11.721769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:11.730467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de5fe-6a48-44bf-b16c-18abaeea24c0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:11.773998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:11.778483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:11.779136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:11.779418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:11.779694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:11.802794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:11.804272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:11.812057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 229/461] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:11.832664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:11.833652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:11.834512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:11.834512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:11.834512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:11.834512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 233/462] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:11.842034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:11.968470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:11.968983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715fcf-966c-45e1-9046-6f673a682e4d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.015733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.022082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.022375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.022654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.023442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:12.023755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.025147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.037939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 230/463] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:12.044123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.089995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.103546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.120779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09f90-b46e-4b32-a683-8d60e0a37c57 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.158860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.159490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.159770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.160040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.171614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.173077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cef02707-1265-4395-8651-8b30357056e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:12.187260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:12.188179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:12.189006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:12.189006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:12.189006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:12.189006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 234/464] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:12.190030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312569-1315-4a7b-a65a-592d99995d9e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.190335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312569-1315-4a7b-a65a-592d99995d9e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.190810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68312569-1315-4a7b-a65a-592d99995d9e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.192432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 235/465] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:12.193182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68312569-1315-4a7b-a65a-592d99995d9e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.197226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47348556-b7ae-4ddb-b6c8-d0c55e369ebe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.234852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.235389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.235667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.235938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.236581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.238041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.261033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 231/466] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:12 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:12.262057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.327048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68312569-1315-4a7b-a65a-592d99995d9e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.371138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.371895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.372250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.372618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.375358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.385573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.387747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.389891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-498c95a8-25e4-478b-bc86-4c71c31cdc3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:12.406256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:12.407336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:12.408590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:12.408590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:12.408590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:12.408590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 236/467] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:12.409446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d869e-9570-463a-a354-f3d135fe4311 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.409616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.410018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.410363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.442677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:12.444358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:12.457446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.457937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.458218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.458490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.459140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:12.459447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.460811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.473355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 232/468] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:12.474354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.528972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 237/469] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:12.530269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.602416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af093a-6618-4e4e-b080-faa1850cc079 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.639638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.640714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.641067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.641413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.642161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:12.642534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.644098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298902-38e4-422c-aa37-0491612356fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.652262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.661677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 233/470] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:12.661677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec32a-5e32-452c-9453-3a66f5123330 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.661677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec32a-5e32-452c-9453-3a66f5123330 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.661677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dec32a-5e32-452c-9453-3a66f5123330 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.662811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 234/471] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:12.666641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dec32a-5e32-452c-9453-3a66f5123330 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:12.672594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500f0ba-8ce3-48bc-903f-adaadca4ce44 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.709562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:12.729236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.730661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.731146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.731643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.732499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.734281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116cdd-6b96-40e1-bb0c-b6cdaf3da1b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.760108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 238/472] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:12 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:12.762440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.762986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.763487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.785458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:12.787250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:12.787563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dec32a-5e32-452c-9453-3a66f5123330 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:12.827774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:12.828408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:12.828740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:12.829020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:12.829720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:12.831175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:12.871531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 239/473] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:12.874564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 235/474] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:12 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:12.918335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.036889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fba435-8714-444f-9171-c40e454c3e21 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.049319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:13.076003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.076643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.076924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.077200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.088376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:13.090455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:13.106658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:13.107969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:13.108816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:13.108816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:13.108816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.112575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 240/475] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:13.113998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.188689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:13.204567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.239399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855491ec-d9b2-4a54-a07a-a2e011079aa9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.285030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.286094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.286572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.287108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.287960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:13.288478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:13.290092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:13.311909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 236/476] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:13.316787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2efb-3ebd-4fd9-8394-ed637f25f543 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.317555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.352040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.352656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.352934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.353207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.353905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:13.356359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:13.371369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 21:42:13.371369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 21:42:13.371369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [parameters: {'id': '944b632247e04968b4cba3e3fbc17cb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 21:42:13.371369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 18 21:42:13.395456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return ref.to_dict() Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:42:13.396363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 21:42:13.397270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 21:42:13.402973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:42:13.403922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:42:13.404719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'id': '944b632247e04968b4cba3e3fbc17cb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:13.405501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 21:42:13.407395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.409569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 241/477] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 21:42:13 2022] POST /identity/v3/roles => generated 144 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:13.416972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.441395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3394848-9512-44e4-9ff2-92682024c06a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.457949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:13.477916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.478917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.479410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.480554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.481393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:13.483339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:13.486432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:13.509657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 237/478] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:13 2022] POST /identity/v3/users => generated 249 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:42:13.591070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740686-016b-4402-9c40-429e4f8ff0f8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.631840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.632877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.633367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.633866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.647369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d382aaf9-dc38-42d2-a5f5-15a95203e7ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:13.653589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:13.654560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 21:42:13.655365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:13.658068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 242/479] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:13.664106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:13.775401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16dc79-bdd5-4314-a77c-e2f27f560ebb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:13.812109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:13.813099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:13.813577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:13.814169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:13.815106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:13.815625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:13.817248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89b2d7-61e1-4fae-babb-c9793cc271bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:13.826772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 238/480] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:14.692637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40097-dd3a-47e6-a156-44919823e26d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:14.693683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40097-dd3a-47e6-a156-44919823e26d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:14.694166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40097-dd3a-47e6-a156-44919823e26d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:14.695935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 243/481] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:14.702166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f775-1624-4232-990d-691fd526797f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:14.702584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f775-1624-4232-990d-691fd526797f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:14.702758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f775-1624-4232-990d-691fd526797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:14.730898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-11e9f775-1624-4232-990d-691fd526797f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:14.732101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-11e9f775-1624-4232-990d-691fd526797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:14.772959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:14.831263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 239/482] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:15.254815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40097-dd3a-47e6-a156-44919823e26d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:15.318534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41045b47-54e6-4b07-b753-233cbe8eb6b3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.319186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41045b47-54e6-4b07-b753-233cbe8eb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.319691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41045b47-54e6-4b07-b753-233cbe8eb6b3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.321502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 240/483] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:15.326727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccae2b5-ea37-4074-b387-4f21dec9d946 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.327261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccae2b5-ea37-4074-b387-4f21dec9d946 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.327745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccae2b5-ea37-4074-b387-4f21dec9d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.356742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:15.362984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0ccae2b5-ea37-4074-b387-4f21dec9d946 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:15.364471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0ccae2b5-ea37-4074-b387-4f21dec9d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:15.377849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40097-dd3a-47e6-a156-44919823e26d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:15.436202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:15.437168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.437651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.438113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.452893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e2df1400-cc55-4de5-b483-e9606d8b5af1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:15.457761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:15.458659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:15.460068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 244/484] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:15.460966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e524-f878-463a-92e9-b66ed5b71322 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.461119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e524-f878-463a-92e9-b66ed5b71322 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.461511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e524-f878-463a-92e9-b66ed5b71322 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.467278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 245/485] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:15.467278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010e524-f878-463a-92e9-b66ed5b71322 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:15.478844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 241/486] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:15.480745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.481831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.482621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.505924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:15.507409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:15.598169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 242/487] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:15.638142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010e524-f878-463a-92e9-b66ed5b71322 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:15.690364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:15.690364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.690364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.690364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.690837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:15.691089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:15.692423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:15.706292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 246/488] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:15.717366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:15.747261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:15.792337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:15.804827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:15.837471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c02cc9-8c31-47d1-b9bf-1aa516a28686 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:15.881538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:15.882569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.883480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.883960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.897345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:15.899894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:15.908734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc107f-b66f-474a-b908-b84e596a6914 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:15.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:15.921784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:15.922629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:15.922629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:15.922629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:15.923597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 243/489] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:15.931348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:15.967304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:15.967935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:15.968216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:15.968489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:15.988094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:15.993295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:15.994232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:15.995048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:16.001545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 247/490] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:16.008128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.068673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.098871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0047-c06c-4123-8565-351c1cec146b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.115891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.139463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.140068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.140353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.140627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.141681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:16.142051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.143766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.157673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 244/491] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:16.158577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.164401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f539fa-b40b-42b9-ba26-1566be20358c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.204200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.204798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.205112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.205409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.206209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:16.206509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.207900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.216211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 248/492] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:16.216916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.272202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f53960-aae5-4e5d-bb33-caaf47c599ae None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.308674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.309310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.309591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.309892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.311789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.322605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea08361-e0dc-47dd-adbf-b6bf09df3ba7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.323561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-763113d2-58b8-4b33-b9d7-9446f9103c3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:16.329251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:16.330294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:16.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 245/493] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:16.332391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd81b00-3bb0-460a-adfb-7534fc5a4ce4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.332679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd81b00-3bb0-460a-adfb-7534fc5a4ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.332957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd81b00-3bb0-460a-adfb-7534fc5a4ce4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.336268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 246/494] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:16.336268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd81b00-3bb0-460a-adfb-7534fc5a4ce4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.355854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.362135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.362968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.363240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.363513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.371603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.373051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:16.385067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:16.386053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:16.386939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 249/495] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:16.390398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.440054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd81b00-3bb0-460a-adfb-7534fc5a4ce4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.472528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.473145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.473425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.473740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.483944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.485380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.488798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2217-594e-4a7d-9a67-ae3262638dda None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b5561803-c50a-4855-83d5-89104c0b185a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:16.500751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:16.501869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:16.503081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:16.503081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:16.503081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:16.503081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 247/496] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:16.503560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f4f53-1e07-4db3-921d-6a70f5c59020 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.503739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f4f53-1e07-4db3-921d-6a70f5c59020 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.504006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f4f53-1e07-4db3-921d-6a70f5c59020 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.505365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 248/497] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:16.506233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b748be-474b-435a-a103-061fec3720ab None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b748be-474b-435a-a103-061fec3720ab None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.506767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b748be-474b-435a-a103-061fec3720ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.524134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-66b748be-474b-435a-a103-061fec3720ab None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:16.525331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-66b748be-474b-435a-a103-061fec3720ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:16.527266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.528122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.528406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.528677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.529339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:16.529675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.531060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.539269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 250/498] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:16.540056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.569016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.593994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 249/499] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:16.594891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b748be-474b-435a-a103-061fec3720ab None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.595835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.661922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aebe9-73cc-4006-a749-3864d598e1a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.700966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b748be-474b-435a-a103-061fec3720ab None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.701919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.702606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.702958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.703307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.704113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:16.704486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.706418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03be29-b568-4011-ac8c-89ee9b9e6662 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.739062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.739783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.740051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.740317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.741011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:16.741313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.742700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.745674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 251/500] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:16.751271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.751563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.751846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.754647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 250/501] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:16.755681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.773844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:16.775042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:16.801215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.849722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 252/502] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:16.849722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.882554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c2499-6711-4ac4-a0fc-befa8cb49d38 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.889339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:16.920697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.921327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.921616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.921905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:16.933685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:16.935177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:16.947696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3066a-bdd4-4b69-ab2b-5b12727e4fe2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:16.950267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:16.951120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:16.952365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:16.952365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:16.952365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:16.952365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 251/503] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:16.952597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:16.986729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:16.987329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:16.987602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:16.987870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.050492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.062902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.063066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d0ac3-6f48-445f-a0bd-4cf4c8b3cbd9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.064939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.098753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.099384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.099661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.099992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.107805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.109310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.114103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:17.122261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:17.123268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:17.124140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 252/504] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:17.124694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.174299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 253/505] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:17.175144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.228421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8ded-6a09-4d71-bcec-4fef167fff35 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.262595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.263226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.263507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.263791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.273225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa918-d436-4e8d-94cd-e43275921aa1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.274747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.276222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.284492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:17.291167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:17.292668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:17.293493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:17.293493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:17.293493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:17.297865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 253/506] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:17.298564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.307582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634abf-d611-4142-91e2-22020c7124de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.308748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634abf-d611-4142-91e2-22020c7124de None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.309034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634abf-d611-4142-91e2-22020c7124de None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.309309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634abf-d611-4142-91e2-22020c7124de None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.309985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634abf-d611-4142-91e2-22020c7124de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:17.310294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634abf-d611-4142-91e2-22020c7124de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.311683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634abf-d611-4142-91e2-22020c7124de None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.325823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.333956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 254/507] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:17.333956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634abf-d611-4142-91e2-22020c7124de None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.405177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08a106-09c7-4fd2-ae9f-625e1bf379c2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.427656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634abf-d611-4142-91e2-22020c7124de None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.437842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.438471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.438755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.439028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.449560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:17.455080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:17.457830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 254/508] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:17.458645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.462629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.463234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.463509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.463776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.465059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:17.465389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.466867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.485229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 255/509] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:17.486070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.519999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.542245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.561235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb82e6-d4c3-4171-a1e8-5281a2b4eda5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.582979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b7ed8-3a28-496a-8deb-bb79b48b1495 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.594579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.595198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.595476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.595747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.606616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.608061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.619418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.620020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.620322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.620591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.621227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:17.621528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:17.622076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:17.623107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:17.623968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:17.623968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:17.623968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:17.624128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.624194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 255/510] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:17.624257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.634596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 256/511] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:17.635458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.735033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00e41f-e2e7-46f7-a00f-fdf7a9546998 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.737130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d56f1-50d4-479e-bec1-ee523115f948 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.747675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.754127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.770074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.770790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.770892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.771204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.771486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.771599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.771760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.772026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.772682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:17.772980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.774371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.780324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.781791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.782842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 257/512] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:17.783628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:17.798119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:17.799212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 256/513] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:17.805178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.881713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d33893-bf66-435f-9d22-19be65be18b7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.901069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65616f-f6f9-452d-b854-4dd7ef323843 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:17.916764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.917405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.917717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.917986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.935838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:17.936444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:17.936725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:17.936998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:17.937660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:17.937969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.939357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:17.951257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 257/514] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:17.952033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:17.967163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.981857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:17.987284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:17.988749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.051336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59149-f4d4-4129-93c5-565988277eb7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.084915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.085525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.085888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.086164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.086587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 258/515] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:18.086803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.087424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.088248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.110585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 258/516] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:17 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:18.111564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.180980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.205442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748469f-1646-4fb1-83d8-55f1de18b69c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.240398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.248971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.249846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.249953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.250229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.255139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe4ed-5bf3-47a0-a9d3-c9afcb34e605 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.262344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.264043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:18.284625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:18.286103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:18.286948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:18.286948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:18.286948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:18.286948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 259/517] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:18.286948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.294479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.295200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.295556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.295902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.296653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:18.296963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.298968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.338001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 259/518] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:18.338879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.401277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6601879-4f55-434f-bf1b-f08ea7c3f4de None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.436766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.443095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.443717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.443995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.444263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.447098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241168d0-7e3d-4ae9-b0c8-8ae75eccad37 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.451922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.453389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:18.468752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:18.469707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:18.470560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 260/519] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:18.471598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.492494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.493090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.493315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.493587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.505039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.506491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.513962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:18.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:18.522967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:18.524009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:18.524009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:18.524009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:18.530925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 260/520] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:18.531572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.583172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496436c-0fcc-414a-8297-0f3e1f07b56b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.621902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.622793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.623075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.623349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.623996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:18.624304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.625701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.642551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 261/521] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:18.642551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.652165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd485c7-2b59-4755-975a-fa333e7baf7e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.693489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.694139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.694421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.694695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.700070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.753300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1d9b-dbef-423a-89f4-2cba8deea7d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.766817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.770613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.772159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.792588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.793289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.793575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.793879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.794573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:18.795098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.796318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.819637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 262/522] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:18.819637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.881144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 261/523] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:18.886391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.931422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9c28-bdfd-416a-94d5-124f008bdc98 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:18.946825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.971178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:18.971793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:18.971793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:18.972358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:18.972957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:18.973270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:18.974691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:18.987068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 263/524] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:18.987916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:18.993173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:18.997502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104ba2-6a56-4a54-8d11-2a9cc3a30aff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.033431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.034100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.034382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.034655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.041774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.043232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:19.054594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:19.055647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 262/525] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:19.056516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.102480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52675b6b-37a1-4e75-b3f9-bdb2de724b02 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.138136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.138749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.139029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.139302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.140029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:19.140337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.141798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.147927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 264/526] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:18 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:19.148733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.179600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeddcc-397d-4de7-896e-25bbed85a7e9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.191753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.238912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.239998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.240586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.241909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.259070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.260573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:19.277395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:19.278786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:19.279631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:19.279631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:19.279631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:19.279631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 263/527] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:19.280303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.288476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f99b8-aa0b-4441-9db9-702cb80baa8a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.294045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.337953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.339592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.340099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.340568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.349486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.351448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:19.364557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:19.365460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:19.366737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:19.368591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 265/528] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:19.369640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.426159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd68-f75c-4e31-8cc4-9aadd3e5b98f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.482123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.483116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.483407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.483684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.484437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:19.484748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.487057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.490833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.499432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac238f-d1c2-463f-a075-063da0d04adc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.519924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 264/529] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:19.519924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.542900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.544002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.544679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.545151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.546097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:19.546600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.548226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.563316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 266/530] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:19.564543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.607204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.651061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516677-db47-446b-a56f-3efbfb007498 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.688969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39438caa-9777-4f4d-9ec8-fce6bcab3c9e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.701553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.702563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.703040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.703527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.704418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:19.704932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.706600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.729848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 265/531] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:19.730995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.740316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.741805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.742599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.743376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.787964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.850062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.852301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.885281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4ce3-93e1-43f4-abb2-b1121321a001 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:19.921626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:19.933345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:19.934027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:19.934475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:19.934812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:19.943789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:19.945712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:19.962157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:19.963669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 266/532] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:19.966076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:19.977685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 267/533] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:19.978944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.038653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:20.085317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015cd6c-66a9-42d1-80eb-cdf63341440f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.097637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d97-4a99-460e-a0c9-2a12926b8fef None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.121999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.122948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.123420 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.123885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.124775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:20.125297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.127022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.140035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.140465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 268/534] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:20.140654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.140936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.141249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.142030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=c9b6922a858f459cbbaf1d5cad9967d1) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:20.142423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.142525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.143910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9321-7a2d-4bcb-af13-c38fd2ee5c5b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.150052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 267/535] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:19 2022] GET /identity/v3/role_assignments?user.id=c9b6922a858f459cbbaf1d5cad9967d1&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:20.151310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a809b-1443-4806-ba5b-f1e2071aa56c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.152148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a809b-1443-4806-ba5b-f1e2071aa56c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.152427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a809b-1443-4806-ba5b-f1e2071aa56c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.157644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 268/536] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:20.157644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a809b-1443-4806-ba5b-f1e2071aa56c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.227404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:20.275337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365ef1-445a-4ec9-a0da-f81af844b2d4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.285442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a809b-1443-4806-ba5b-f1e2071aa56c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.296517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:20.318763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.319720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.320140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.320552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.328496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.329250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.330227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.331287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.332662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=71ebfbeddd1943f6af2c0229edbbc41d) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:20.333098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.334454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.335105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125d8d-ea54-41b5-8503-cd6bdb5d1845 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.336084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.349630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 269/537] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/role_assignments?user.id=71ebfbeddd1943f6af2c0229edbbc41d&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:20.349630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.349630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.349630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.364564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:20.365543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:20.366612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:20.367619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:20.367619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:20.367619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:20.368037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:20.369660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 269/538] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:20.375640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.452369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 270/539] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:20.570846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263ca9-2a39-40ef-8f81-0fe7b258fac2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.610586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.611191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.611473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.611747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.612391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:20.612696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.614135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.627506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 270/540] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:20.633056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.691223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:20.772867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51706fca-86e9-4375-ad84-23f3f7eb7fbb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.788233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.817709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:20.818489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:20.818778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:20.819054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:20.827051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:20.828853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:20.842053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:20.843097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:20.844229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 271/541] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:20.853287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:20.859069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:20.980851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70d5d-4164-4b91-a944-39cdb469cbcf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:20.990236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:21.023131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.024127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.024608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.025077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.026006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:21.027500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:21.029751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:21.030392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedf010-ef3d-4a3f-98b5-ee0856397779 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:21.044962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 272/542] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:21.055285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.094021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.094711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.094992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.095629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.113762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:21.119865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:21.121288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:21.122117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:21.122706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 271/543] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:21.127444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.258334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:21.270793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884c198-c099-4a21-bca2-9380dbe7b7de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:21.285851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b83a3-1a6d-4253-89d0-ea3fa71ef992 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:21.324717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.325430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.325757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.326129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.326809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:21.327112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:21.328515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:21.337821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 272/544] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:21.338474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.339738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.340016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.340287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.341032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=2837aa09ce6d4000985f475688f612e0) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:21.341331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:21.349660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cede183-d221-4723-b23e-5278f3f98036 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:21.350930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.353728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 273/545] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/role_assignments?user.id=2837aa09ce6d4000985f475688f612e0&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:21.420542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b353-e9b7-42e5-938f-c251656dd669 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.420859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b353-e9b7-42e5-938f-c251656dd669 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.421100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b353-e9b7-42e5-938f-c251656dd669 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.422724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 274/546] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:21.432154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.432455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.433277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.467081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:21.468697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:21.525248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:21.567218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec01e0-44d0-4c65-875b-f0380402c029 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:21.612739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 275/547] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:21.635164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.636424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.636701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.636970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.656011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:21.657493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:21.672410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:21.673315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:21.674180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:21.674180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:21.674180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:21.674180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 273/548] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:21.679383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.821404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a61ed1-c3c1-47b3-87aa-303480121a5c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:21.874307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:21.874931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:21.875210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:21.875476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:21.876116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:21.876413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:21.877828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:21.892545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 276/549] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:21.898172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.901092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:21.905678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:21.930142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.073657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d75fe-ef24-4576-b1cb-13c588b2ef33 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.079079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc124-ac13-41f4-8daf-b675c159fa03 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.122841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.123838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.124397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.124871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.133440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.135621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:22.149114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:22.150606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:22.151687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:22.154726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 274/550] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:22.160581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.180701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.181720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.182208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.183353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.195589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:22.202287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:22.203240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:22.204062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:22.207395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 277/551] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:22.212638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.304485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.374400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee05046-4c7c-44d2-8f11-e0c78cd6d426 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.403182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.443485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d0629-dd70-4cbf-b676-3b3f848b6041 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.454320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.454954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.455232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.455508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.456261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:22.456568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.458122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.488707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 275/552] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:22.498658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.506583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.507193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.507469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.507734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.508399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:22.508696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.510097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.519456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 278/553] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:22.524708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.631109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d035de-be56-4b84-bde2-6cd9c22603a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.640498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adba28-7a00-42a4-ad07-76f910d9e272 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.652552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.667501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.668107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.668387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.668664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.677265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.678089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.678375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.678658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.690505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.692265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.694294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7a643c9b-8123-499a-99ec-7cddf6044f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:22.709792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:22.710692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:22.711549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:22.711549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:22.711549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:22.711549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 279/554] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:22.712374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0fcc-124f-4984-a91f-a1acca43d638 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.712661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0fcc-124f-4984-a91f-a1acca43d638 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.712935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0fcc-124f-4984-a91f-a1acca43d638 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.714216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 280/555] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:22.721890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6c1b-6fdf-4504-9290-429e06dff178 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.721890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6c1b-6fdf-4504-9290-429e06dff178 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.721890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6c1b-6fdf-4504-9290-429e06dff178 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.721890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 281/556] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:22.721890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b6c1b-6fdf-4504-9290-429e06dff178 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.750378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.751872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c88eb-c193-419f-b826-177e73a69c7a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.833457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b6c1b-6fdf-4504-9290-429e06dff178 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:22.857950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 276/557] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:22.864078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.864373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.864650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.871747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:22.872362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.872648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.872979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.873580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:22.873915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:22.875310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103fb7a-a2ca-43fd-b8d6-e45d269c2c0c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:22.884740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:22.886897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:22.888704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 282/558] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:22.890016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:22.890305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:22.890583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:22.909250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.911607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:22.913591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:22.970173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:22.977662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 277/559] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:22.977662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:22.988768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 283/560] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:22.989723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.098204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab4615-1a2a-40e1-8e4c-7c156f0189e3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.132436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613c386-2f01-4679-9d7d-8167ee0489f0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.139886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.140522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.140801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.141070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.153027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.154589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:23.170489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.171444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:23.172322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:23.172322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:23.172322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:23.172322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 278/561] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.177340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.181337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.186248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.186902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.187186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.187460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.206452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.208380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:23.222869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.223824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:23.224772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:23.234028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 284/562] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.239262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.308587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.373055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbf6ac-bd05-4617-ba07-632cf583e9b9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.373404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b964383-e12c-4204-ab4a-585f95acd46e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.412919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.413502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.413814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.414145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.414807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:23.415111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.416590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.416716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.417230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.417512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.417815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.418457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:23.418761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.420138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.432394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 279/563] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:23.433841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.437378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 285/564] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:23.438307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.493292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.533145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd3a89-5a70-45a4-bf84-243faa44e4e1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.535292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd877b-c6bf-4bee-9bb8-7809dbc4c842 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.536798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.568679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.569307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.569589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.569894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.570187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.570817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.571106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.571405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.580726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:23.582037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8e4118d7-08b6-47f4-87fd-36f1b55231e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:23.585398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.586444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:23.587262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 280/565] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:23.587927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.588815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:23.589649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:23.590288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580b52-cf7b-4498-bd8b-d35b048d0289 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.590359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 286/566] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.590359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.590476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580b52-cf7b-4498-bd8b-d35b048d0289 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.590476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84580b52-cf7b-4498-bd8b-d35b048d0289 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.591393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 281/567] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:23.593647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84580b52-cf7b-4498-bd8b-d35b048d0289 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.692890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7795c-632e-4094-bc60-34fe90e6f2d5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.694039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84580b52-cf7b-4498-bd8b-d35b048d0289 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.718450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.728278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.728911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.729194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.730025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.731057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.731701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.731980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.732254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.736717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.738201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.742258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.743695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:23.749705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.750572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:23.751612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 287/568] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.752365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:23.763367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:23.764291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:23.765156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:23.765156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:23.765156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:23.765156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 282/569] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:23.765372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.778844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.852722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce6d8-133f-49d1-964a-a734682243d1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.867999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f822d-5ad1-4153-bfd3-1660dbf42142 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:23.886228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.886834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.887116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.887389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.888055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:23.888357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.889753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99d998-3099-4dbd-9311-19217fcabb52 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.897417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 288/570] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:23.898678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.898968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.899245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.905036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:23.905659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:23.905935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:23.906202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:23.906861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:23.907160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:23.908686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:23.917380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:23.917678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 283/571] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:23.918473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:23.919340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:23.952407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:23.984363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 289/572] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:23.985193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.003154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.027100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387daa6-b948-4333-8ec6-c18bb1fae110 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.064857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.065897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.066456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.066933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.068259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.068774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.071144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.087375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 284/573] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.089313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.092981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611f393-1e90-465a-a67d-3d3f7c657886 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.129317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.130269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.130558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.130834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.131483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.131788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.133154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.148765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 290/574] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.148765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.204993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.212133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eadaf-4133-4b02-911a-a550e09beb34 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.225464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.253068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.253945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.254226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.254499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.255548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea39ee5-a7e3-4679-92f8-59ae1fe13de5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.265166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.266636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:24.281914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:24.283002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:24.283876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:24.283876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:24.283876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:24.283876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 285/575] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:24.284108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.291165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.291791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.292683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.292951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.303165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.304604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:24.318327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:24.319466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:24.320401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:24.320401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:24.320401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:24.320401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 291/576] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:24.320401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.385844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87bc2e-cbe6-461d-a3da-ed241aafc054 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.419325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.419928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.420209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.420482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.421002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7135-72fb-4e04-a64c-b6f8c8aa1d62 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.421210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=156f961161644bc6a2f19e0b1a0a32c4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.421521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.422992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.428534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 286/577] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/role_assignments?user.id=156f961161644bc6a2f19e0b1a0a32c4&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.429316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.445265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.449222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.460435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.461004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.461403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.461709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.470580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.472157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:24.486515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:24.487523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:24.488412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 292/578] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:24.488887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.551490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c9cd4-6b6d-48a4-badd-ab2e650ef732 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.591683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.592687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.593162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.593675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.601353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6a10d-058c-4767-859e-97f254b12edc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.605424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:24.610817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:24.611933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 287/579] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:24.612758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.644424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.645130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.645474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.645852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.646497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.646800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.648181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.667035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 293/580] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.667035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.695059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.722366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.734100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db139d6-e1a2-45fb-b138-3d5bdb056b8e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.778144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.778645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f431-7acf-4a06-8963-7bb37fedd7a5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.778925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.779276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.779619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.780500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.780879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.782967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.795029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 288/581] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.795657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.822295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.822901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.823823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.824103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.824787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.825089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.826524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.839308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 294/582] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.840142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.904457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ed38-1451-4d99-8567-64cbaa6bebeb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.936338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.943041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.943847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.944201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.944644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:24.945418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:24.945858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:24.947568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:24.950904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fd20c-e261-4c64-90d2-95e8cfc5d221 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:24.961758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 289/583] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:24.961758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:24.962018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:24.992276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:24.992831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:24.993114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:24.993386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.002834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.004982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:25.017989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.019129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:25.020280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:25.026716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 295/584] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.026716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.076928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dd322-a887-4f16-a0e0-b807120d6465 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.111864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.112612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.112965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.113313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.121421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.122970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.126751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579f0fc-615f-4760-ab91-78f37b3a2f8f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:25.134243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.135178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 290/585] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.141731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.170525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.171180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.171463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.171743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.187680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=d441753da1af4f3c8f3b5fb4088d201b, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.189155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.194578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.202658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 296/586] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:25 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:25.203627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.234575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.276096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32266da-bb8d-43c2-aa52-4a03129a5d55 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.333033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.333742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.334027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.334298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.348215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c8cc-be04-4cb1-bb08-609699c9e64c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.348794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.350590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:25.367287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.368261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:25.369149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:25.369149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.369149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:25.369149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 291/587] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.369824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.395829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.396639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.396917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.397187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.398632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:25.398632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.399651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.418637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 297/588] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:25.419483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.496540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0d4f-9feb-488d-89e7-758c13049335 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.504891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.531881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.537625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.538241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.538554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.538774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.539804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:25.540109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.542404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.563761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 292/589] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:25.564714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.567666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bb3d3-4483-4e31-ade7-f1862f6cd8d3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.616024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.616398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.616675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.616945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.617586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:25.617916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.619275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.632290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 298/590] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:25.633148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.717708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be811d-6281-4ed2-a5bd-b321f8951fd7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.753135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423bec1-0987-4bf3-a857-e539cb169900 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.759940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.760577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.762025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.762315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.774897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.777102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.790978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.791538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.791814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.792142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.794142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:25.794276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.795359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:25.796315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:25.796315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.796315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:25.800491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 293/591] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.801721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.810993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.812795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.822797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:25.830750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.831854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:25.832843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:25.832843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:25.832843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:25.835095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 299/592] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.836057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:25.924232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caf11-ee7f-4693-83fd-944b989defed None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.960138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1d4bc-5d1d-4ff0-ab0a-1a75d9dc647c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:25.961534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:25.962836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:25.963271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:25.964368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:25.975478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:25.977167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:25.989990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:25.991109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:25.992058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:25.994786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 294/593] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:25.996135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.008640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.009254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.009532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.009845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.011382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.011689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.014621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.033677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 300/594] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.034820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.087669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.109185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.141310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab127aa7-3b62-47c4-b6aa-672293d32c43 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.148884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdb80-9cd3-4c12-ac05-77c122d7c508 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.189027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.189720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.190000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.190280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.190552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.191075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.191223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.191506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.191829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.192262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.192885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.193499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.194004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.195745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.209268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 295/595] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.212967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.219098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 301/596] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.220074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.330145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9979-022e-4efb-8e08-03d372ed7249 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.355576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccaed4-1f9e-4512-9a7f-63e33da0bb58 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.359281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.367937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.368568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.368840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.369107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.377648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.378977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.380754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:26.393799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:26.394916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:26.395806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 296/597] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:26.396357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.408254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.409994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.411094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.411876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.421861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.423915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-845511b6-d1ef-4007-87a7-90da8e76fd29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:26.437054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:26.438092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:26.438976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:26.444481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 302/598] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:26.444481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb733-6088-418d-b89b-469a5ffaaa1f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.444481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb733-6088-418d-b89b-469a5ffaaa1f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.444481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb733-6088-418d-b89b-469a5ffaaa1f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.444481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 303/599] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:26.446336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efb733-6088-418d-b89b-469a5ffaaa1f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.502818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fae024-afef-4712-98b8-a29afe71dc37 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.537779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.538438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.538723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.539002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.550207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.551850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.556467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efb733-6088-418d-b89b-469a5ffaaa1f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:26.567457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:26.568506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:26.569547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:26.569547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:26.569547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:26.571190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 297/600] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:26.572123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.597340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.598182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.598650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.598919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.599603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.599905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.601307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa172f-79c5-4cb1-affc-1f0b1be181e2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.618880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 304/601] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.618880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.618880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.618880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.628078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.636730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:26.638084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:26.660445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.683933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aadc8-2a40-46f6-a166-964b9883e5eb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.723081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.723514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.723861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.724195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.724957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.725328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.726908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.737937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 305/602] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:26.737937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.739660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 298/603] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.740470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.855273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4aa74-7c11-4e2f-9a5f-ae3a930831af None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.878509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288024-d7b0-4236-9312-a7d960221a0c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:26.889186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.900017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.900970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.902102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.902565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.904162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:26.904786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.906725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.918505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:26.919133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:26.919607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:26.920030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/c9b6922a858f459cbbaf1d5cad9967d1/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:26.922612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 306/604] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:26.923653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:26.943217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=c9b6922a858f459cbbaf1d5cad9967d1, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:26.944195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:26.947000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:26.966986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 299/605] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:26 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/c9b6922a858f459cbbaf1d5cad9967d1/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:26.968748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.067046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445cc-b1e8-44fe-be13-c2b01d365ca3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.110984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af27d-fd4b-4e3a-8896-fdd3ef441d61 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.114522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.115205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.117743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.118146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/71ebfbeddd1943f6af2c0229edbbc41d/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.139049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=71ebfbeddd1943f6af2c0229edbbc41d, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.141043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.152413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.153321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.153625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.153921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.162141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.163629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.165150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 307/606] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:26 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/71ebfbeddd1943f6af2c0229edbbc41d/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:27.166365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:27.176791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:27.177867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:27.178723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 300/607] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:27.184781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.212519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:27.257577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:27.311011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37155c-c771-4768-bfc7-0829462f94c6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.339755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b5592-3f65-4b00-8f96-01c6b6bbdbce None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.351973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.352651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.352894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.353169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.366352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e8c7418f-891f-4a66-af76-a49fb57482b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:27.371556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:27.372571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:27.373398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 308/608] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:27.374626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae1b0-16b2-4cd8-ad13-57bd4b2a0b7e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.374626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae1b0-16b2-4cd8-ad13-57bd4b2a0b7e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.374626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae1b0-16b2-4cd8-ad13-57bd4b2a0b7e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.375802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 309/609] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:27.380912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.381200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.381473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.390353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.390902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.391175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.391444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.392142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:27.392457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.393945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.405749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:27.406967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:27.409407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 301/610] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:27.410316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.495061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:27.502739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 310/611] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:27.503987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.541428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a791d0e-ab17-4bfb-8d0a-88664516ff8a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.570261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:27.621641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d3463-757a-467d-81ad-5a45c62d8291 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.659404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.660145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.660536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.660918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/2837aa09ce6d4000985f475688f612e0/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.670324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.670958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.671239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.671511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.672254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:27.672560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.674039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.677316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=2837aa09ce6d4000985f475688f612e0, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.678970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.699064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 311/612] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:27 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/2837aa09ce6d4000985f475688f612e0/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:27.699998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 302/613] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:27.707259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.882506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:27.911027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a140c-0369-4fa9-bae5-9ba08763d1ad None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:27.913564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:27.964206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:27.965446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:27.965777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:27.966169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:27.977504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:27.979472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:27.996260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:27.997426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:27.998563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:27.998563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:27.998563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:28.000057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 312/614] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:28.005556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.009341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721c5e-d9d6-4072-a7aa-2c7f97d720dd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.046664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.047536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.047963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.048382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.056482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:28.068709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:28.074118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:28.075200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:28.076027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:28.085349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 303/615] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:28.086255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.129419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbe8f4-1193-4000-9902-5cbc3700cd03 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.167801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.168743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.169216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.169711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.170584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:28.171149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.172705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.186662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 313/616] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:28.193016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.213352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:28.306876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47908-00a9-42fc-afd4-4b4cedef6f96 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.362802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de358a-aa6d-43dd-a6dc-49d05b4d5122 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.363800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.364802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.365330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.365840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.374913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.376628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:28.390151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:28.391113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:28.392070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:28.394874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 314/617] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:28.400462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.448792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.449817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.450365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.450853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.453241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:28.454530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.456136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.475874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 304/618] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:28.481625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.526749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:28.555828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b52e04-06c5-457f-a362-fd5a2ab0a6db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.597784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.598769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.599295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.599773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.600706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:28.601214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.603041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.624984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 315/619] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:28.631057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:28.633106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.732151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840afd5-f454-4843-994b-7de93fd8d4c6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.753652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681702d0-9d8a-45ca-8a49-b8e30ca3a4fd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:28.798142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.798780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.799043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.799308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.818797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.820265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.827629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:28.828258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:28.828530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:28.828795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:28.831600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:28.843946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:28.844938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:28.846298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:28.846298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:28.846298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:28.846298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 305/620] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:28.850966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:28.852472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:28.854010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:28.887411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:28.888447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:28.889334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:28.889334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:28.889334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:28.893767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 316/621] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:28.899510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.034480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571d53-a1dc-435d-a0c7-70c4c271c521 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.069967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.102640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.103273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.103552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.103829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.104507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:29.104812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.106218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.115832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739082-0de9-4f8b-8251-689742b499e5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.120218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 306/622] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:29.125440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.223277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.223910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.224196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.224476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.225129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:29.225437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.226990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.235126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f349b-8c8a-46aa-8269-7fbf1f3dc78a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.253982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 317/623] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:29.262735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.296106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.296664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.296942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.297211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.315072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.317531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-692bc4bd-bf36-4c3f-afd0-8fadf10cf632 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:29.332632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:29.333661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:29.334598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 307/624] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:29.335663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b1df4-7173-434c-b2ed-08dafa81b275 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.335663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b1df4-7173-434c-b2ed-08dafa81b275 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.335909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b1df4-7173-434c-b2ed-08dafa81b275 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.341218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 308/625] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:29.342561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.344272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b1df4-7173-434c-b2ed-08dafa81b275 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.390274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.406776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f14392-ab79-45e1-bc95-d39a667968a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.448788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.449433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.449750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.450025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.457021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.458506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-94fa31ca-5ffa-41bc-8756-3568aaad1c6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:29.470226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:29.471213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:29.472102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 318/626] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:29.472662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.472736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.472983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.480920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b1df4-7173-434c-b2ed-08dafa81b275 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.492000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:29.493284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:29.516613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.517543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.518046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.518314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.519056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:29.519377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.520795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983889b1-6317-430c-b8b2-731ade898724 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.545771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 309/627] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:29.547526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44063ac-be3c-42bb-b05b-aab29660e580 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.547786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44063ac-be3c-42bb-b05b-aab29660e580 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.548067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44063ac-be3c-42bb-b05b-aab29660e580 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.549351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 310/628] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:29.554083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44063ac-be3c-42bb-b05b-aab29660e580 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.567002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 319/629] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:29.568091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.610973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.683324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5eea5-98c7-4c5e-b291-c68689e175f1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.698070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.705240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44063ac-be3c-42bb-b05b-aab29660e580 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:29.727232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.728045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.728368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.728775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.763343 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:29.764820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.765713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.766647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.767641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:29.768605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.772369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a4140-8db3-4312-be42-6324d05f05fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.793721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 311/630] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:29.795499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:29.795940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:29.796600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:29.811245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:29.812798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:29.820532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:29.822522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:29.850631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:29.905985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 312/631] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:29.905985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.931858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 320/632] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:29.933727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:29.998299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.018192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aba24c-a15f-4638-8cb2-183f0e7906e6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.043321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daeca-9e26-4e19-8720-9e13924a438e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.055953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.056612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.056893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.057162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/156f961161644bc6a2f19e0b1a0a32c4/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.074563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=156f961161644bc6a2f19e0b1a0a32c4, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.076125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df13ff-0da1-4ef6-8c2e-ee659f0236b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.088519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.090138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.090667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.091142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 313/633] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:29 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/156f961161644bc6a2f19e0b1a0a32c4/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3962c-abfe-472f-842c-80fa882634ec None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3962c-abfe-472f-842c-80fa882634ec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3962c-abfe-472f-842c-80fa882634ec None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 314/634] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:30.105124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3962c-abfe-472f-842c-80fa882634ec None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.106707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.117216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-875be415-da91-41b9-91ce-56db247d0739 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:30.124390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.125448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.126335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:30.128547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 321/635] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.130141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.131058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.131724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.160262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:30.162008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:30.244620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 322/636] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:30.246835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.251290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3962c-abfe-472f-842c-80fa882634ec None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.287450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.294427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.295484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.296135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.296409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.307999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.309652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:30.328163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.329112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:30.330101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:30.330101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:30.330101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:30.330101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 315/637] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.330388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.372008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd47ec-4a89-4a4b-b13d-d7b249d4d1b5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.404919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.426187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.426807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.427083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.427353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.428020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:30.428320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.429723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.439573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 323/638] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:30.440423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.451368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995e1e2-9ac4-47ea-b610-cc56d777ee7e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.495014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.496508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.497105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.497579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.510123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:30.517166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.518325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.519228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:30.525750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 316/639] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.527525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.558842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fb3b6-16c9-48f2-ae36-52d21f03df8a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.581482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.595958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.596582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.596868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.597150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.597828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:30.598136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.599505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.611987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 324/640] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:30.613130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.645238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef003c-6f82-44d5-afda-87ff3875435d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.671838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.684448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.685086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.685367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.685676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.700690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.702203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:30.721141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.722092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:30.722993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:30.722993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:30.722993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:30.722993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 317/641] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.723212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.734006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f2c54-541d-4ac9-90fd-e71a7b11de9f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.775397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.775932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.776307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.777077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.777779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:30.778343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.779706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.792621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 325/642] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:30.792621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.835241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.843191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed7c5-ed96-4584-a0cf-ef8391729c4e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.880269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.881250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.881536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.882149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.892740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:30.897924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.898892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 318/643] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.899773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:30.909233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3648c-d348-44f8-8af7-5c404866782d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:30.920277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:30.947894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:30.949239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:30.949496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:30.949758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:30.957110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:30.958602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:30.970021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:30.971115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 326/644] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:30.972156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.013072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f64b1-e732-40df-bfdd-ba069ccba9eb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.049470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.050120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.050399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.050666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.051308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.051610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.052985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.065794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 319/645] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.066686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.089276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995260-4a7a-42fe-8200-6b801f3b1d37 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.089651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.131095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.131733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.132021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.132297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.143601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.145072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:31.160158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:31.161163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:31.162072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:31.162072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:31.162072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:31.162072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 327/646] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:31.162072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.192341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feeabe-962a-46b0-ba60-1b19fd420cd4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.192739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.232356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.233319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.233884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.234158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.234827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.235135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.236525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.246455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 320/647] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.247099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.280239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf635148-6a5b-4b29-a53e-fbfc22352cce None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.317582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.318251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.318494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.318769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.319478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.319775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.321241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.338355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 328/648] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.338355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.342649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.366812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c0f4-9abc-4dd2-921e-202cfa6c95fd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.402543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.403426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.403702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.403976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.411568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.413013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:31.425037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:31.426025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 321/649] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:31.426947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.435472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.456128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2420f5-4899-4f44-b796-dd629f8995e8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.493449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.494179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.494463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.494737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.496319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.496319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.497790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.509225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 329/650] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.514345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.539608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed9627-40f5-4285-82d4-d9a9b7729edd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.577354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.578011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.578287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.578580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.586147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.591724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.593533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:31.609362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:31.610610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:31.611485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:31.611485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:31.611485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:31.611485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 322/651] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:31.611485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.623221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa30923-c891-419e-86ea-08dcb1345a7b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.660825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.661854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.662423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.662763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.663576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.663967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.665687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.681955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 330/652] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.683368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.696577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.731998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c75f4c-78a2-4784-affa-0b83b1d98258 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.771208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.772234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.772720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.773198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.774349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.774896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.777119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.803908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 323/653] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.805296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.810251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9fb67-fd92-4e11-9230-789419b790a0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.853721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.854900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.855191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.855575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.864734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.865187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.867907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:31.882921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:31.883897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:31.884865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 331/654] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:31.885451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:31.933055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53d1a-048c-4235-a59e-566589eba10a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:31.965000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:31.980144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:31.980766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:31.981047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:31.981323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:31.982018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:31.982326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:31.983719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:31.995407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 324/655] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:31.996233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.008165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea0b3-8adb-405d-aec3-3668580b89cd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.045746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.046793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.047262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.047866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.121680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.123558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957ea9c-b1e4-42cc-9705-cdfdb9b0dc42 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.128325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1558a-2c08-4ac8-b3d6-e5f1903ff429 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.140317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.172122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.172854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.173202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.173549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.174446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:32.174823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.176359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.194513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.197483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 325/656] 10.222.0.82 () {56 vars in 1137 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:32.198273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 332/657] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817237-49f8-4fda-952c-97bb081f1618 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817237-49f8-4fda-952c-97bb081f1618 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817237-49f8-4fda-952c-97bb081f1618 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 333/658] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:32.241441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817237-49f8-4fda-952c-97bb081f1618 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.310180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57004873-965d-4e16-af0c-b6ae88e532e1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.346960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.348028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.348196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817237-49f8-4fda-952c-97bb081f1618 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.348647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.349093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.357316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.359058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:32.371949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:32.372866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 21:42:32.373841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:32.380279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 326/659] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:32.381320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.384420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.385042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.385320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.386157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.410515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.421481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.458287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.459760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.487678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f5380-d60d-42dd-9e26-48aab557c963 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.525565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.526834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.527175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.527535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.540808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.542926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b18188d9-899f-4507-9b5a-652b83488ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:32.556625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:32.557834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:32.558961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:32.558961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:32.558961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:32.562454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 327/660] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:32.563660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.563952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.564238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.569047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 334/661] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:32.570112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.586370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:32.587657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:32.641049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.659013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 328/662] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:32.662502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.676267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.684232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188655-ea0e-4d14-a256-54563e7bcc72 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.740128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.740867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.741241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.741638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.742469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:32.742904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.744480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.763439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 335/663] 10.222.0.82 () {56 vars in 1139 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:32.764375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.800760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed6acf-3c9d-4b76-b946-019c50ecfac6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.848389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.849571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.850018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.850454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.851506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:32.851981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.854792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.872006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 329/664] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:32.873029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cce195-5382-49d3-842c-cfe6c45f2b80 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:32.873268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.898775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:32.912214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:32.913064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:32.913423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:32.913856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:32.927437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:32.931096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:32.949919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:32.950889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:32.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:32.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:32.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:32.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 336/665] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:32.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:32.965965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.004028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0da23-fe66-4cf6-ab5e-66d4e117f94e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.044029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.044696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.044980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.045255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.062591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2aab9e-fc91-4a60-8b91-cb1126b36f6a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.101535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.102214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.102493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.102766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.111287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.113031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.120990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.122602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:33.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:33.128712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:33.129838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:33.133717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 337/666] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:33.134321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.177319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.194260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.232104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 330/667] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:33.237313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.248705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087090-3e07-4f8a-b477-2a7dcebb9139 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.286746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.287380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.287656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.287923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.298577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:33.303614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:33.304546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:33.305383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 338/668] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:33.306139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.346537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebc26-2df5-42aa-b3c9-7b4efbf83d49 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.382600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.383213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.383492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.383768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.384407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:33.384709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.386678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.402496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 331/669] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:33.402496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.406954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.413942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56757d-e022-47cd-9394-e7c1aac6b6e2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.431858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.455170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.455783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.456057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.456329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.457018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:33.457321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.458786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.470949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 339/670] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:33.471755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.524094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1765e62-caf2-49ac-a873-fc9ecbd17748 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.560200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.560831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.561105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.561370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.571919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.573378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.580023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c10783-c388-49d8-a6ab-45dbe01aad08 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:33.588737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:33.589939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:33.590910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:33.590910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:33.590910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:33.590910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 332/671] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:33.590910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.622630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.623242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.623517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.623784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.624444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:33.624743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.626132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.634807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 340/672] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:33.638814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.668002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.678866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.703778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb751fe-ab31-4591-8659-caee0faa9b75 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.752526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.753157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.753433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.753728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.761806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.763267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.763537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3bcdd-cd60-4b13-a40c-ba79af6493b5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:33.775032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:33.775945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:33.776863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:33.778596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 333/673] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:33.779861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.802075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.802775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.803131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.803479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.804204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=c9b6922a858f459cbbaf1d5cad9967d1) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:33.804516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.805970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.812271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 341/674] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/role_assignments?user.id=c9b6922a858f459cbbaf1d5cad9967d1&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:33.812898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.900159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1160-fa7d-47bc-818e-95811443e426 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.907478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.931022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f05a-84ae-4800-a600-b64428fce676 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:33.942449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.943414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.943697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.943966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.944613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:33.944911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.946347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c76e-ec4c-427c-b1cc-60d7ce79b1f3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:33.957375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:33.962082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 334/675] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:33.963333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932dab-55ba-4b2d-85fa-0fe1fef7a286 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.963646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932dab-55ba-4b2d-85fa-0fe1fef7a286 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.963927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4932dab-55ba-4b2d-85fa-0fe1fef7a286 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.965265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 335/676] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:33.970026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4932dab-55ba-4b2d-85fa-0fe1fef7a286 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:33.980683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:33.981909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:33.982299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:33.982694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:33.996301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:33.997974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:34.012909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:34.013946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:34.014841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:34.014841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:34.014841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:34.014841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 342/677] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:34.015500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.085013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4932dab-55ba-4b2d-85fa-0fe1fef7a286 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.124778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.125702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.125981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.126248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.127361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:34.127673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.129552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d42839-58f8-4d04-89e6-fedf6c1b987d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.138701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880c97d-779e-464a-807d-be17cb810f93 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.148158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 336/678] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:34.150013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.150376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.150726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.169918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:34.171217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:34.178980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.179728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.180013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.180289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.182493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.188774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.190453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:34.202868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:34.203787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:34.204661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 343/679] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:34.205138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.237339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.250679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 337/680] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:34.258926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.316937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae6ba-0bb5-43aa-a741-bcd556bf223d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.363442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.364064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.364347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.364624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.365681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:34.365996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.367373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.381909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773341da-c22c-47bd-b3df-0d91a5cf8e28 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.386110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 344/681] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:34.387259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.432337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.433198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.433485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.433799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.434542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=71ebfbeddd1943f6af2c0229edbbc41d) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:34.435315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.437169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.450179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 338/682] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/role_assignments?user.id=71ebfbeddd1943f6af2c0229edbbc41d&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:34.450179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.461815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.525781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.560720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c255e30-9a47-4845-ad01-90c65cc81a63 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.579938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23a63-0fd7-4bc7-854a-d5c9bd4c1687 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.607570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.608193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.608474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.608749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.609455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:34.609818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.611270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.620374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.620374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.620374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.620958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.629787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 345/683] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:34.630139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.630724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.631791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:34.647206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:34.648442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:34.649720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:34.650344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 339/684] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:34.651799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.777843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6651b9-7b72-40ea-8812-c17f33765db0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.780213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537dee4-4ca9-41b4-8b74-c4fbc5937b4f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:34.792858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.801774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:34.815342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.816219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.816569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.816763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.817493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=2837aa09ce6d4000985f475688f612e0) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:34.817881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:34.819371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:34.825785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:34.826303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 340/685] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/role_assignments?user.id=2837aa09ce6d4000985f475688f612e0&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:34.826857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:34.827103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.827536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:34.827971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:34.849858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:34.856678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:34.857952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:34.858838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 346/686] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:34.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:34.962725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315fa8-0904-4ba0-88e4-62bee76fe2e2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.008962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.009701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.010050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.010388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.011774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:35.012146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.013671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.029498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16990cc-f238-4099-872a-c525d6aa57b3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.064308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 341/687] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:35.069955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.078356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.082049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.082436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.082805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.083594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:35.083997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.087185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.100246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 347/688] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:35.107388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.126875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.157841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.204007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c81f4-9753-4516-8f42-ab992ee4b123 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.228832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987ec7-34d2-49ca-a70d-bde1873484a1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.242590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.243958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.244388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.245819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.269387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.270194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.270767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.271064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.283240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.286082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:35.305377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:35.306576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:35.308278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:35.308278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:35.308278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:35.309901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 348/689] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:35.318757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.337777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.340049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.417931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.430968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe78d3e-0115-4397-a10a-10821ad52a60 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.459930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.472837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.473491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.475308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.475591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.476248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:35.476552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.477952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.492209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 349/690] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:35.498349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.618748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c873-6ae3-42dd-87ca-076db27243cd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.658441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 342/691] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:35.663472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.686532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.687524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.688060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.688535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.699887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.703865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:35.726526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:35.727503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:35.728423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 350/692] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:35.734216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.754807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.864822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1a817-5df5-41a4-8e37-a3c04bbaff36 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.901583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216b3d1-3978-4656-9968-a4cafa130f14 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:35.903533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.905121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.905482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.905858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.908885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:35.920129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.922195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.940569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:35.941223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e584629e-722b-4400-b80d-5fefe58350b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:35.941381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:35.942345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:35.943220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:35.943220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:35.943220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:35.943397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.943397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.943397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:35.943397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:35.943647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 343/693] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:35.944347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57467e7-0c9d-4bdc-8803-76c2d3345fcd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:35.944441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ac38-0051-4b90-9b8d-c72cd33e3354 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.944839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ac38-0051-4b90-9b8d-c72cd33e3354 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.945385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ac38-0051-4b90-9b8d-c72cd33e3354 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.948791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 344/694] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:35.949652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ac38-0051-4b90-9b8d-c72cd33e3354 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:35.965130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 351/695] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:35.967637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:35.968624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:35.968898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:35.987779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:35.989177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:36.052820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.061940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 352/696] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:36.062984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.072524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ac38-0051-4b90-9b8d-c72cd33e3354 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.110330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.111201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.111694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.112124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.112939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:36.113523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.115135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.128627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 345/697] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:36.134661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.173057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53870-cf94-4ccd-b0a5-d51b6ac78966 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.229664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.230620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.231238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.231512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.231783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.304842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7882-10eb-4eae-8603-ff6741e6d10d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.349995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.351017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.351509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.351992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.357973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.364889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.366595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.367742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.369187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:36.385181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:36.386753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:36.388241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 346/698] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:36.392746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.478019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 353/699] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:36.478921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.506676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbf049-ae13-47fd-861d-4919c48c2062 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.516760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.545820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.546463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.546740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.547012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.547708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:36.548015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.549474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.567836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 347/700] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:36.568737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.599372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.606680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e734b6-b875-4f2c-8b3d-6baeb03ed0b9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.645831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.646506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.646785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.647059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.647648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.649022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:36.666844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:36.667879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:36.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 354/701] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:36.671354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.699399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650f01-49fa-48d5-8044-348b435e94d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.749443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.750027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.750304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.750571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.765408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.766901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:36.787994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:36.789000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:36.789911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:36.789911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:36.789911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:36.789911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 348/702] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:36.790700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.808401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa36-d368-4e4d-a5cd-ce301a8b906a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.819089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.847552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.848548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.849020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.849469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.850407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=156f961161644bc6a2f19e0b1a0a32c4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:36.850975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.852593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.860007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 355/703] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/role_assignments?user.id=156f961161644bc6a2f19e0b1a0a32c4&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:36.861135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:36.898253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:36.915798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cbc08-806d-4b38-9bf8-1b7f9e5fc878 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:36.962091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:36.963435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.964401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.965181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.967220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:36.967755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:36.969346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd60ae-435b-4fdd-82c3-2d4441147a47 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:36.981633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 349/704] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:36.981633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448fc9e-2c8e-47ff-984f-9df28729e92c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:36.981633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448fc9e-2c8e-47ff-984f-9df28729e92c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:36.981633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448fc9e-2c8e-47ff-984f-9df28729e92c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:36.982531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 350/705] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:36.988557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448fc9e-2c8e-47ff-984f-9df28729e92c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.014857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74f965-33b6-48e9-9102-e42dca608b92 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.063520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.064666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.064666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.064843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.065468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:37.065818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.067206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b224-b32a-48e2-bc06-a5a029e0b9f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.082464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 356/706] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:37.084547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b6c2-9836-4bcd-b25a-c6e7d81e6331 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.084830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b6c2-9836-4bcd-b25a-c6e7d81e6331 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.085099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b6c2-9836-4bcd-b25a-c6e7d81e6331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.095689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:37.105324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5cd7b6c2-9836-4bcd-b25a-c6e7d81e6331 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:37.106557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5cd7b6c2-9836-4bcd-b25a-c6e7d81e6331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:37.122686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448fc9e-2c8e-47ff-984f-9df28729e92c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.163108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.163736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.164019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.164290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.164931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:37.165232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.166837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.173761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 351/707] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:37.174601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.226299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 357/708] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:37.297209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd7c6c-814e-4ab1-a223-d85a7ccd5b4d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.324612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ea3f7-8274-48f3-90b5-66e67b915234 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.324963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ea3f7-8274-48f3-90b5-66e67b915234 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.325172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ea3f7-8274-48f3-90b5-66e67b915234 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.326813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 358/709] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:37.332327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.332612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.332889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.361302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.362174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.362460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.362736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.364200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:37.365432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:37.375225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.376729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.386201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:37.397952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:37.398941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:37.399875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:37.403218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 352/710] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:37.406633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.425951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:37.477041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 359/711] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:37.553200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1a8f7-d95d-4b14-844b-8da7fff6608b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.572843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.593331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.593980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.596442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.596725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.597461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:37.597783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.599254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.620440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 353/712] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:37.628042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.772986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:37.792319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fa0ee-993f-4ab9-aab5-166ccc823d7d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.809427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b271e-5057-43c2-acd0-cf0a56b2ea2d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:37.839976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.841016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.841564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.841874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.845396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:37.846383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:37.846821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:37.847246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:37.848110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:37.848558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.850169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.852802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:37.854862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:37.859713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 354/713] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:37.859890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.861491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:37.878596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:37.879616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:37.880520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:37.880520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:37.880520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:37.884678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 360/714] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:37.886877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:37.997989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ee835-9da3-4676-9e5c-c27f4f9133a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:38.013032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ee99a-e98b-4622-b543-8ddb1a712cb6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:38.052815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:38.061866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:38.063883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:38.064845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:38.065676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:38.067041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:38.067905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:38.069555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:38.070411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:38.070911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:38.071189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:38.071850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:38.072158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:38.074096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:38.090176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:38.091196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:38.092027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 355/715] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:38.094296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 361/716] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:38.097105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:38.101054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:38.217982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:38.233994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc4ac-e7c6-47e7-8867-d0b26f2aca07 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:38.272521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:38.273948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:38.274458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:38.274943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:38.275782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] RBAC: Adding query filter params (name=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:38.276349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:38.278551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:38.283888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 356/717] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:38.289956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:38.354054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:38.356811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f053eb-8068-4905-ac4a-216d9eac748f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:38.417517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d388d-6043-459b-b9bc-35b32e849d44 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:38.436923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:38.437948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:38.438451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:38.438922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:38.439785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:38.442902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1abcf-138c-4c25-ae72-9fcdba15b8e4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:38.487928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 362/718] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:42:38 2022] POST /identity/v3/users => generated 254 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:38.490127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:38.491061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:38.491545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:38.492074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:38.492950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:38.493533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:38.495964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a073a0-132e-4078-b8c5-42829e5a126e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:38.502285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 357/719] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:39.055214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d1eb-bc52-4415-a1a3-7fb3c4730325 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.055726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d1eb-bc52-4415-a1a3-7fb3c4730325 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.055823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d1eb-bc52-4415-a1a3-7fb3c4730325 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.057079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 363/720] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:39.069185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be7fd-852b-4694-8bd7-c798bb11160d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.069469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be7fd-852b-4694-8bd7-c798bb11160d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.069767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be7fd-852b-4694-8bd7-c798bb11160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.090818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-273be7fd-852b-4694-8bd7-c798bb11160d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:39.092004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-273be7fd-852b-4694-8bd7-c798bb11160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:39.184151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 358/721] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:39.571354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252d1eb-bc52-4415-a1a3-7fb3c4730325 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:39.621239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7303c-4f9d-4c26-a8a5-cb74dc5b968a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.621797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7303c-4f9d-4c26-a8a5-cb74dc5b968a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.621797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7303c-4f9d-4c26-a8a5-cb74dc5b968a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.623455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 359/722] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:39.637315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.637633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.637919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.667883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:39.669093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:39.706718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252d1eb-bc52-4415-a1a3-7fb3c4730325 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:39.725987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:39.746654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:39.747409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.747615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.748053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.759715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:39.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:39.766122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:39.766957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:39.769375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 364/723] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:39.779191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:39.892863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311de48b-93c0-4501-ae2c-0ad99a05cc03 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:39.932191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:39.933173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:39.933765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:39.934220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:39.935069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:39.935568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:39.937227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:39.947039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 365/724] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:39.952911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:39.990659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:39.992424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:40.051259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 360/725] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:40.072147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff217b8b-d5f3-4ac4-b486-c3bfdda81def None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:40.113260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:40.114256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.114756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.115241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.126775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:40.128525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8fc4726b-c77a-449a-8458-a74fc8f4d413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:40.143848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:40.144846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:40.145755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:40.145755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:40.145755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:40.147845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 366/726] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:40.154584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:40.283585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cab98c-fe40-49dc-937c-58204516b9de None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:40.331558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:40.332598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.333134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.333692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.334544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:40.335045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:40.336724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:40.340034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf0922-1d79-46b7-a038-f84c4dc2af45 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.340555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf0922-1d79-46b7-a038-f84c4dc2af45 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.341029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf0922-1d79-46b7-a038-f84c4dc2af45 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.342829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 367/727] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:40.348059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.348623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.349107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.374984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 361/728] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:40.379372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:40.380881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:40.391758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:40.468806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 368/729] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 21:42:40.469980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:40.481983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:40.488485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:40.537062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454fbf5-ee4c-4f4b-991d-109a4630dc86 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:40.602257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:40.604777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.606236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.606727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.611740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b10375-61db-442e-bc98-c8476ccd833b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:40.630848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:40.633043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:40.652191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:40.653452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:40.654401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:40.664500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 362/730] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:40.683023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:40.685471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:40.689898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.695268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.697386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.698362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:40.701135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7638d-b575-4fb7-8cf9-575040776036 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:40.732285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 369/731] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:42:40 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:40.738367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69013-a403-438e-ad78-e359aba682cb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.738997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69013-a403-438e-ad78-e359aba682cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.739540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69013-a403-438e-ad78-e359aba682cb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.741392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 370/732] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:40.749400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.749873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.750277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.772269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:40.773798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:40.861961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c24e0-d610-49f3-bd3d-22407348c814 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:40.866266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:40.888446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 371/733] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 21:42:40.889406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:40.892600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:40.938497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:40.939332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:40.939464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:40.939718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:40.940455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:40.940762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:40.942721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:40.983561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 363/734] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:40.986892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.022407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d4951-696c-45e3-98b5-85ad250d59ff None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.060550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.061761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.062240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.062773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.074985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:41.080818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:41.081766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:41.082587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:41.089779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 372/735] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:41.091451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.103327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b22156-d022-4c85-a881-34bdda160518 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.161419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.162398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.164091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.164586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.199658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.218872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.250244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275aa76-963e-409e-8649-abbe1215540b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.263641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.265109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.293021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.293670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.293958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.294230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.294926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:41.295234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.296618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f892749-a981-4a92-91df-9c4daf06b992 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.310917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 373/736] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:41.310917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a413-7abe-4bcf-bf58-58eb44a88dc2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.310917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a413-7abe-4bcf-bf58-58eb44a88dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.310917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a413-7abe-4bcf-bf58-58eb44a88dc2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.313734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 374/737] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:41.314535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139a413-7abe-4bcf-bf58-58eb44a88dc2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.374913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 364/738] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:40 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 21:42:41.376052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.433313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139a413-7abe-4bcf-bf58-58eb44a88dc2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.440681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.467107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.479839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.480456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.480738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.481011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.481689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.482519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6026b2d-dd43-4a94-a6fe-5180b29b0522 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.483063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddf6cc-f32d-49e8-baa0-2a1ac3582c23 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.497876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 375/739] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:42:41 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:41.499046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.499334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.499606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.520845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:41.522151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:41.547795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.548859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.549153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.549425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.567414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.569814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:41.589975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:41.591015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:41.591910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:41.591910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:41.591910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:41.593747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 365/740] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:41.594922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.623993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 376/741] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:41.624943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.723185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2140-62b5-4fd5-bda3-4dfaf152444e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.749419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.766721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40020f-de0e-4193-a149-de1aad73b9ba None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:41.769031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.769982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.770484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.771014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.782634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.784369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:41.800627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:41.801874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:41.802806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:41.802806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:41.802806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:41.804854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:41.805048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 366/742] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:41.811637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.839451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:41.840078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:41.840361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:41.840639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:41.843001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:41.843315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:41.844684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:41.866266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 377/743] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:41.872687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:41.976887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e04c3-c923-4058-81f4-2a88bd17f103 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.018057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.018932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.019216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.020384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.021104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:42.021358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.022771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.039916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 367/744] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:42.040783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.062218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b7754-f169-4198-8b29-91285768f0fe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.071518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.105543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.106297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.106558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.106870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.116623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.118216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:42.130278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:42.131231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:42.132153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 378/745] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:42.132724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.166319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54741ee-f9ac-4318-a766-882384212b45 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.182900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.221030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.221963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.222439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.222782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.237051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:42.244286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:42.246249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:42.247089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:42.249742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 368/746] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:42.251075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.272937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c362-3a35-4c31-af4b-ed4561fab376 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.309766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.310785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.311268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.311741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.319309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.321051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:42.332639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:42.333510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:42.334459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:42.336470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 379/747] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:42.337443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.348999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.372455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91f37-467a-4208-a37d-4a562b9ab6c4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.410993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.412297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.412297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.412297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.413052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:42.413366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.414880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.439265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 369/748] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:42.440168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.440769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.445525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780decb0-7445-4c33-82ad-a66188fa2ee5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.481870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.482870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.483368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.483845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.484727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:42.485250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.486882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.496623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 380/749] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:42.497775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.568116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c849f6-8ce8-451a-a8ec-81c1b2121e83 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.616482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.617511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.618545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.619032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.620278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:42.620965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.623292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.631123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b8c17-0c15-4b33-8785-228a072c3d2b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.637820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.641969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 370/750] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:42.643016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.685028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.685898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.686961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.687694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.711656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.717328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:42.736285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:42.737286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:42.738217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:42.738217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:42.738217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:42.738217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 381/751] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:42.738904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.752438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:42.783217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418fb55-feae-45ba-b375-2cac36425556 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.833713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.834764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.835250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.835723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.851939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.854637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.867553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d108be-83f2-41c7-af08-fe353325488f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:42.875378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:42.876345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:42.877245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:42.877245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:42.877245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:42.884242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 371/752] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:42.884242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.914968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:42.916089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:42.916681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:42.917159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:42.918199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=156f961161644bc6a2f19e0b1a0a32c4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:42.918775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:42.920636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:42.927525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 382/753] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/role_assignments?user.id=156f961161644bc6a2f19e0b1a0a32c4&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:42.928674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:42.961983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.020759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.045116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9d48c-5cad-46eb-93f1-831524058296 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.064740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4074cc-d51e-4f62-ba99-6aea8148400f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.098794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.099946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.100296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.100639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.101693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:43.102068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.103897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ecbf-5c27-460d-9c5f-b18d67109c27 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.110640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.111895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.112434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.112898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.113783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:43.114311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.115890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712ba23-99a8-4467-8e76-8fd75f7a5663 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.120918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 372/754] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:43.123058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327ebf-ea08-4869-abaf-f46ff46c1ec8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.123349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327ebf-ea08-4869-abaf-f46ff46c1ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.123619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b327ebf-ea08-4869-abaf-f46ff46c1ec8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.125358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 373/755] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:43.130961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b327ebf-ea08-4869-abaf-f46ff46c1ec8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.136630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 383/756] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:43.138250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.138773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.139252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.159001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:43.160579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:43.242349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 384/757] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:43.243774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.316563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b327ebf-ea08-4869-abaf-f46ff46c1ec8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.317527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.347099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.380505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cf55e-a426-49e4-a26b-c528c030b3fa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.399931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.401043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.401544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.402029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.418363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.419015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.419303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.419578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.426823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.427797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.428823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.429249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:43.444668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:43.446437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:43.447567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 385/758] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:43.449952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:43.455524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:43.456601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:43.457746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:43.464284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 374/759] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:43.465786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.568697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54efc-8bb1-4488-bb67-94d2748a38fe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.602604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.614174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.614934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.615221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.615498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.616244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:43.616557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.617669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164f5f-80f0-4ece-967d-9ae62fa10884 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.618033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.645700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 386/760] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:43.646864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.655764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.656863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.657355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.657849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.658749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:43.659257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.660900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.679071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 375/761] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:43.680650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.721214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.762576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536104d1-c0c6-4230-95b9-44cf982798fd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.797428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed049-34a4-40ee-8e92-954c3c55d38f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.800389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.801019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.801292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.801559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.802170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.803537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fd8af0b4-6f60-4f9e-8b1a-d6d62c177369 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:43.812376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:43.813609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:43.814653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:43.816710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 387/762] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:43.818026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fdf4e-e1a8-4e4d-8c71-0ef2b30911ed None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.818461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fdf4e-e1a8-4e4d-8c71-0ef2b30911ed None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.818880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fdf4e-e1a8-4e4d-8c71-0ef2b30911ed None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.820258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 388/763] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:43.821108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fdf4e-e1a8-4e4d-8c71-0ef2b30911ed None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:43.848502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.849109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.849655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.849931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.869632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.928451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.929837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.931729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fdf4e-e1a8-4e4d-8c71-0ef2b30911ed None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:43.961354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:43.970785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:43.971596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:43.971875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:43.972144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:43.974456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=d441753da1af4f3c8f3b5fb4088d201b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:43.975909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62161e17-6305-4e0b-a1d6-f2c45bdc5cd5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:43.993176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 389/764] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:43 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:44.003127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.003957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.004471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.034694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:44.036071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:44.053545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 376/765] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:42:44.054679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.125237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 390/766] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:44.126760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.175559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.187294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680cff-3686-4230-bcef-341a8cee4185 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.230404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.237463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.238226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.238521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.238871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.239556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:44.239861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.241703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.248212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 377/767] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:44.254186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.257161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caba58-c345-4531-afe8-1a0e5440580a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.321262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.321939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.322224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.322499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.342804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.345104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:44.370265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:44.371255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:44.372788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:44.372788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:44.372788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:44.372788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 391/768] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:44.380205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.439982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0a0a-8d40-4e1b-a9c3-8ce1d7f7a9cc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.486451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.487067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.487352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.487630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.488261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:44.488569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.490158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.501696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 378/769] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:44.503215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.518446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.536438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a2fd7-2654-4181-a51c-4a24fc7eb8e0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.573164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.576527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.577176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.577434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.577740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.578845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:44.579121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.580794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.595919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 392/770] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:44.596874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.627260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6d9f7-814e-4263-8350-86c3d1eb5d71 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.670121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.671382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.672445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.672944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.674023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.676761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:44.688850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:44.689890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:44.690752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:44.699021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 379/771] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:44.699021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.718881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee5887-604d-4a0a-923d-441057ba1f71 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.756954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.757968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.758445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.758906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.759914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] RBAC: Adding query filter params (interface=public, service_id=51c04c35ae3442489d39b06ce9ffa56f, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:44.760496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.762100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.770489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 393/772] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/endpoints?service_id=51c04c35ae3442489d39b06ce9ffa56f&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:44.771719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.788504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.824419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11c6d-9253-4eee-a619-0f3144654922 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.855657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:44.870561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.871335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.872151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.872521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.882656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.884769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.891659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78663-b890-45a5-aeab-9e9cf08de2a4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:44.900026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:44.900969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:44.901895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 380/773] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:44.903305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:44.933662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:44.934866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:44.935329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:44.935783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:44.936596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:44.937105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:44.938680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:44.943685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 394/774] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:44.944699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.030182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:45.052545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f82cb-148b-449f-978e-2d37c2c71e3b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.059969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6559c3-4931-40e4-ab7a-e6ef11e279d1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.097115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.097531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.098165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.098440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.098709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.099053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.099402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.099738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.100221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.100760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.101426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:45.101957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.103722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.111210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 381/775] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:45.111210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.131824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 395/776] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:45.138066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.172259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:45.238688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243f6fc-9347-42cd-be9c-de4c7793bc49 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.284381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.285122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.285407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.285707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.286366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:45.286890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.288803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57db33d-803a-4fd2-bed2-4b3bde458a3f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.297461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7d55-2fbf-4b99-8ca9-7e734d64a3bd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.300482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 382/777] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:45.301815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54217b-ec5c-41f7-a93f-64bede7ce1f8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.302104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54217b-ec5c-41f7-a93f-64bede7ce1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.303774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54217b-ec5c-41f7-a93f-64bede7ce1f8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.303867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 383/778] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:45.310735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54217b-ec5c-41f7-a93f-64bede7ce1f8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.350988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.351661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.351941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.352270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.353112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=1222623a682a4d85bd967e96b9277098) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:45.353427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.355713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f73d-11da-4d43-a932-687a578f8684 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.357622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:45.363219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 396/779] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:45 2022] GET /identity/v3/role_assignments?user.id=1222623a682a4d85bd967e96b9277098&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:45.364789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.365184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.365515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.388368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:45.389622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:45.470771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 397/780] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:45.517096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54217b-ec5c-41f7-a93f-64bede7ce1f8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.535117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:45.567998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.568623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.568940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.569220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.569967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] RBAC: Adding query filter params (interface=public, service_id=005afc4f53d94b56bff97d073cae4ee0, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:45.570277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.571723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.581956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 384/781] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:42:45 2022] GET /identity/v3/endpoints?service_id=005afc4f53d94b56bff97d073cae4ee0&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:45.587636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.696604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c19f42-1e57-4095-b721-5da6f63e6dec None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:45.734199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:45.734822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:45.735157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:45.735433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:45.736082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:45.737464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:45.738674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:45.743975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 398/782] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:42:45 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:45.820367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:45.964589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602eef6-8f63-4a86-9295-009aa7c8c64c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:46.000920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:46.001686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.002173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.002466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.013417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:46.019932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:46.020833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:46.021693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 385/783] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:46.028610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:46.137415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e532eb4-a748-43f2-be1f-6186e6f5c677 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:46.191286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:46.191902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.192179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.192452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.193122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:46.193422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:46.194809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123e07f-103a-42aa-abc1-06d9fb212244 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:46.208389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 399/784] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:46.214362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:46.251480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:46.345535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34caee-3acc-4e8d-9832-9a44f6543cf4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:46.387816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:46.388439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.388719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.388987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.390103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a60a4-eb48-4e7b-9d55-f6eb4eea2ee2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.390528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a60a4-eb48-4e7b-9d55-f6eb4eea2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.390931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a60a4-eb48-4e7b-9d55-f6eb4eea2ee2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.393203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 400/785] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:46.400560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.401005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.401411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.402057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:46.403837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:46.425654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:46.426859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:46.427369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:46.428276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:46.429151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:46.429151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:46.429151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:46.429904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 386/786] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:46.441229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:46.549506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:46.578594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:46.587126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 401/787] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:46.601415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ba978-34ac-4bdd-a150-9e5b2c3ecdf3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:46.657257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:46.657968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.658245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.658516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.659154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:46.659453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:46.660833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:46.678666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 387/788] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:46.684563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:46.850550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dc25c-959d-4223-90fc-774de4b22310 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:46.895622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:46.896253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:46.896528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:46.896796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:46.906062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:46.907691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:46.922811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:46.924048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:46.925514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 402/789] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:46.929768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.049693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84231d9-247a-4595-9760-90137d8120ef None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.092332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.092949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.093224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.093492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.094255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:47.094556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:47.096154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:47.120811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 388/790] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:47.126656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.174964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.235541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.245488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.250606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3acf6-f178-4210-95be-991641b32d22 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.327954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.328594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.328871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.329141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.353746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:47.357190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:47.381107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:47.382118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:47.383669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:47.383669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:47.383669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:47.383669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 403/791] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:47.388620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.418892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a4e0-ce67-4570-a553-0d54c43447f6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.463691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.464303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.464582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.464852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.488922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:47.493572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:47.494482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:47.495898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:47.506636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 389/792] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:47.514587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.561967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.569545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4639a32-e66c-4359-8af5-3bdf947c4be5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.595161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.614912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.615853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.616516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.617339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.618172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:47.618621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:47.621277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1326f-70c2-44a0-846b-ef825f6b0081 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:47.633073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015abf61-dff3-441c-8418-ac89c98ba87e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.637424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 404/793] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:47.642042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90de61-d31e-44a5-a746-e08497de0f55 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.642042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90de61-d31e-44a5-a746-e08497de0f55 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.642042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90de61-d31e-44a5-a746-e08497de0f55 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.642042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 405/794] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:47.642042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e820-2b32-4164-92d5-aa9089165a15 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.642409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e820-2b32-4164-92d5-aa9089165a15 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.642409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e820-2b32-4164-92d5-aa9089165a15 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.643688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 406/795] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:47.644393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4e820-2b32-4164-92d5-aa9089165a15 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.675796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.676411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.676691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.676964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.677656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:47.677965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:47.679329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0b8b-59e5-4ae5-9130-439c680a5561 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:47.692635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 390/796] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:47.692635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c36e6-2203-41de-b12c-8b72f057050b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.692635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c36e6-2203-41de-b12c-8b72f057050b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.692635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c36e6-2203-41de-b12c-8b72f057050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.707854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d68c36e6-2203-41de-b12c-8b72f057050b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:47.709243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d68c36e6-2203-41de-b12c-8b72f057050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:47.761207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4e820-2b32-4164-92d5-aa9089165a15 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:47.797631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 391/797] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:47.797631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.797631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.797631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.805935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:47.806641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:47.807021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:47.807913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:47.815076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:47.815216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:47.816404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:47.816840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:47.829926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:47.830986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 407/798] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:47.831871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.850062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.890029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:47.908916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 392/799] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:47.908916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:47.989978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdef1-3d15-40b0-8353-87002bf7029e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.025502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.026216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.026495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.026767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.037337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.038823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:48.054537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:48.055519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:48.056490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:48.056490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:48.056490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:48.056656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa39e22-16d4-4efa-88bb-c86886622149 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.057054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 408/800] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:48.066594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.098471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.099173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.099451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.100270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.101325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.101655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.103404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.118891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 393/801] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:48.126710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.185132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.202042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.226264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b6ca-b26a-448e-8ca1-1777f10eb1a0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.254589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a7032-e61b-4f2e-881f-5ac5076a9578 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.261316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.261955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.262238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.262508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.263150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.263458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.264815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.277178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 409/802] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:48.278007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.289075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.289769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.290049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.290320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/156f961161644bc6a2f19e0b1a0a32c4/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.292828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=156f961161644bc6a2f19e0b1a0a32c4, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.294289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd34f0-e466-47f2-baf6-d591bef4814d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.309790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 394/803] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:48 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/156f961161644bc6a2f19e0b1a0a32c4/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:42:48.310991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e0ed-b0f0-457a-b60f-7fdbfb58cf1b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.311275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e0ed-b0f0-457a-b60f-7fdbfb58cf1b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.311542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e0ed-b0f0-457a-b60f-7fdbfb58cf1b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.317247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 395/804] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:48.317247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593e0ed-b0f0-457a-b60f-7fdbfb58cf1b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.405445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822770f5-fb74-40ca-9e14-73aa10046fe8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.432159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593e0ed-b0f0-457a-b60f-7fdbfb58cf1b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.440691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.443193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.449865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.451268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.452594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.452936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.455927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.455927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:48.468143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:48.469170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:48.470080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 410/805] 10.222.0.82 () {56 vars in 1129 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:48.470863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.488845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.489478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.489816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.490069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.501521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-27786457-39d7-4326-b9c6-49ef11f8c7e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:48.506498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:48.507403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:48.508218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 396/806] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:48.508907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967322c-c304-4b35-afdc-35700d680977 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.510262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967322c-c304-4b35-afdc-35700d680977 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.510262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967322c-c304-4b35-afdc-35700d680977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.529203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6967322c-c304-4b35-afdc-35700d680977 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:48.530853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6967322c-c304-4b35-afdc-35700d680977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:48.583060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201ce0-2d7f-475b-ad84-8f743974b7ed None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.612531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 397/807] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:48.613446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967322c-c304-4b35-afdc-35700d680977 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.620761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.621397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.621697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.621968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.629855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.631223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:48.643867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:48.644856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:48.645814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 411/808] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:48.646405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.705964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.752332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.759264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967322c-c304-4b35-afdc-35700d680977 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.764192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d627-c983-438e-a082-6227cf099db5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.804565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.804786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.805554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.805725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.806030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.806346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.806455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.806856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.808003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.808435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.808613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.809053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.810108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.810863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.817666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 398/809] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:48.818638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.825140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 412/810] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:48.831431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:48.932840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19a22e-b5bf-41a6-9fa5-fca2d11c7625 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.939114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bcde-fc3b-4a0a-b9d4-8736b8e9bc38 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:48.953907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:48.975159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.975770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.976062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.976323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.977057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.977365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.978282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:48.978831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.979308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:48.979760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:48.980208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:48.981019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:48.981512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:48.982964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:48.993908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 413/811] 10.222.0.82 () {56 vars in 1124 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:48.995799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.000981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 399/812] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:49.000981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.021111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.104383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6b04-4098-4264-b82e-0ff929177776 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.104976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae82b6-6bbf-431b-9c15-bb1810d92262 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.140638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.141306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.141588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.141956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.143007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.143628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.144072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.144341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.145250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.146680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.152672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.154390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:49.157364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:49.158487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:49.162641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 400/813] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:49.164065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:49.170152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:49.171087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:49.172212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:49.172212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:49.172212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:49.172212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 414/814] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:49.172212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.207153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.267125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.283680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf0edb-f798-43a3-9b40-d04e27f2ca6c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.285996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c36e6c-61c6-4479-81fa-c8c87c1066fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.320559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.321168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.321446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.321781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.325451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.326100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.326381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.326654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.327506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.328715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.353834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 401/815] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:42:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:49.355109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.410268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.412093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.448467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.535389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ef11-b46c-4d49-a5eb-90d7684bf7f0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.543596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 415/816] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:42:49.544774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.583513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.584104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.584385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.584659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.585289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] RBAC: Adding query filter params (name=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:49.585611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.587002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.589827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.593870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 402/817] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:49.594729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.667765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8fa6d-1e1d-4396-9c62-f4447b693272 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.722260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e35583-6929-41cc-9d78-00f33836e25e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.763988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.764963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.765428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.765932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.779340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.781127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:49.797318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:49.798451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:49.799347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:49.799347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:49.799347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:49.801488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 403/818] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:49.802576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.832305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.838709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.839713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.840294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.840583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.841500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:49.842024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.843537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:49.856665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 416/819] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:49.857820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:49.879590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:49.954063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0b3a0-623b-4142-b5ce-626832657355 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.968648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c5de-4d84-4b78-be58-c7215db556a0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:49.994794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:49.995759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:49.996189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:49.996610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:49.997389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:49.997870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:49.999412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.005442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.005758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 404/820] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.006199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.006666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.007002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.007526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.007633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:50.007999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.009370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.030654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 417/821] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.031537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.084023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.117815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598b1e-4f56-47b3-ab7d-28df5784c043 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.135881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.139082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011def-61bd-4502-a130-4520f3cf856e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.157311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.158304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.158789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.159263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.167927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.169685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.176976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.177619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.177909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.178188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.178917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.180401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:50.184140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:50.185092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:50.186054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:50.188601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 405/822] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:50.189937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.208478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 418/823] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:42:50 2022] POST /identity/v3/endpoints => generated 337 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:50.209427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.318193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5c3d2-796c-4b92-8e44-04d17f11ef92 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.359841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba3dac-7b42-4839-b0d1-eb33f662275e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.362004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.370840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.371781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.371781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.372037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.380769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.382568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-972ee918-d57f-4126-9c88-95e330d22fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:50.397217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:50.398222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:50.399110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:50.400359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 406/824] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:50.400359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c899-58cf-464c-865c-99b5f1ec9e46 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.400645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c899-58cf-464c-865c-99b5f1ec9e46 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.401457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c899-58cf-464c-865c-99b5f1ec9e46 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.403009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 407/825] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:50.410521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.410804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.411803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.420540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.421159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.421439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.421745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.422482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:50.422786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.424220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.434997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:50.436289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:50.443925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 419/826] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.444791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.469774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.518789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 408/827] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:50.519454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.563249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3b5e7-c03f-472e-8c57-0353e6a9f216 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.603248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.605148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.605671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.606139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.607142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:50.607639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.609310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.623066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 420/828] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.623798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ddda0-2d75-4351-9795-d2d2ff129252 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.633650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.655888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.660838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.661526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.661838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.662165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.681533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.684299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:50.703638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:50.704612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:50.706181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:50.706181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:50.706181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:50.706181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 409/829] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:50.718571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.743604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.753672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96328321-551a-45a3-8ce7-2d25c4efa91b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.790959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.792380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.792837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.793258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.794218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:50.794678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.796468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.803201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 421/830] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.803201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.835213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbed64-6bad-4785-a9fc-0ee7129f6e56 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.873768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.874465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.874859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.875230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.875987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:50.876392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:50.877938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:50.896953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 410/831] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:50.898114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:50.911349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.932763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f9933-f3f9-447b-87f7-790f8d6fc240 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:50.976662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:50.977311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:50.977660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:50.977940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:50.988996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:50.994503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:51.000557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:51.001480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:42:51.002359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:51.003203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 422/832] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:51.008268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.021803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8abd2-1aea-40b1-b115-b8f079fe73c7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.072246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.073308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.073836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.074792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.085227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.087256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:51.103123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:51.104615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:51.105514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 411/833] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:51.114602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.139935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0cf26-d09c-4583-bc2c-0a857ffbba82 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.184711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.185348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.185695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.185975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.186884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:51.187201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.189360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.202020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 423/834] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:51.210578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.221386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:51.234902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8fc31-2693-45fc-bdd9-49f7a45b9381 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.271262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.271870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.272144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.272410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.273113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:51.273416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.274896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.283851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:51.290723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 412/835] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:51.296942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.359287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7649228-a26e-403f-92c9-148cba46abaf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.395857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.396417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.396698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.396972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.410491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.412480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.418762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29a0e-1636-4891-8228-c24b70025bf5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:51.434001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:51.434991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:51.435852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:51.435852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:51.435852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:51.435852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 424/836] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:51.440971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.464288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.464954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.465230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.465501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/1222623a682a4d85bd967e96b9277098/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.480885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=1222623a682a4d85bd967e96b9277098, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.482340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178edb4-9fa9-4627-b3d6-9896a339cee0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.496933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 413/837] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:42:51 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/1222623a682a4d85bd967e96b9277098/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:42:51.498315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865c9e-77a9-47ee-b13a-56f6e54ee6c5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.498603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865c9e-77a9-47ee-b13a-56f6e54ee6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.498875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865c9e-77a9-47ee-b13a-56f6e54ee6c5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.500342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 414/838] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:51.506057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.506624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.507102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.512772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:51.531860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:51.533176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:51.565165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a124f0e-d7fc-454f-bb0e-b92cb84ae05f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.595015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:51.609618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.610246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.610526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.610801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.611454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:51.611758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.613147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.629767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 425/839] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:51.632418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 415/840] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:51.633753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.754983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014c23-2c53-42a7-ba90-fc3ea5d16c74 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.773778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:51.793970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.794591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.794867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.795134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.803050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.804486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:51.817451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:51.818475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:51.820011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 426/841] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:51.824841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.910303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:51.936266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5cc61-d0f9-4d73-b54d-d5ca89c836e5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:51.981635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:51.982191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:51.982467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:51.982736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:51.983469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:51.983771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:51.985790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.011937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 416/842] 10.222.0.82 () {56 vars in 1135 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:52.018024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.045398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d8cf-266c-4315-b86b-1724ed95e92e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.055256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.082784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.083419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.083697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.083967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.084544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.086031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:52.095374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:52.096383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:52.097195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 427/843] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:52.104044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.139087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c9c9-db73-4b7e-8d93-2f9c0c454727 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.174478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.184505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.185407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.186672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.186952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.231078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ffe9-7ad7-4884-aa72-37f5a010923f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.274921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.277082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.277489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.277763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.278043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.278568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.279199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:52.279502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.281010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.290523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 428/844] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:52.292067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.324679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.396999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 417/845] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:42:52.403052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.407191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef162932-a2b8-4da8-bd8b-6cce4754e44f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.414979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.445566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.446382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.446763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.447137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.447907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:52.448311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.449901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.461625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 429/846] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:52.525689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c77122-af42-4f61-9d0e-3aee15114ba9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.564926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.565579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.565880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.566150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.569199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.579611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.581383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:52.597451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:52.598488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:52.599365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:52.599365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:52.599365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:52.602818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 418/847] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:52.605678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.718598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b2e8-7001-4f37-9528-fc246fd8c60b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.753414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.754068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.754408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.754683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.755393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:52.755699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.757174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85835e97-3db7-43a7-9f23-075896173aec None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.773748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 430/848] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:52.775343 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:52.862596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d573a-fdd8-47ff-a2d5-d1a854c71a57 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.862895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d573a-fdd8-47ff-a2d5-d1a854c71a57 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.863078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d573a-fdd8-47ff-a2d5-d1a854c71a57 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.864669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 431/849] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:52.871157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.871838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.872570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.890088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa74f-e280-4af9-9a09-0c5b08a128c0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:52.896538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:52.898559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:52.931467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:52.933635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:52.934257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:52.934561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:52.934837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:52.944767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:52.946602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:52.962313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:52.963239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:52.964347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:52.966727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 419/850] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:52.972430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:53.000031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 432/851] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:53.032926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:53.086982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb80d7-3986-47a4-8d07-9ee4d74a6658 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:53.123506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:53.124043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.124323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.124594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.125281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:53.125582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:53.127223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:53.142704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 420/852] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:53.152503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:53.264066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7955-2b58-4c25-9b7c-c6ec58063b9c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:53.285315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:53.306400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:53.307129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.307846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.308149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.309446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=156f961161644bc6a2f19e0b1a0a32c4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:53.309796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:53.311412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:53.323136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 433/853] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:53 2022] GET /identity/v3/role_assignments?user.id=156f961161644bc6a2f19e0b1a0a32c4&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:53.411355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ad995-365d-4d60-9585-6f51bd9b881f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:53.423501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:53.452409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:53.453440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.453985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.454506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.455498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:53.457266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4197d20c-5e13-461a-85b5-1ada51b75006 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:53.470073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:53.471254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:53.472156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 421/854] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 21:42:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:53.480237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:53.510720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:53.569289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c472-d43e-4aac-8c0b-367b5b310899 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.569664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c472-d43e-4aac-8c0b-367b5b310899 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.569884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c472-d43e-4aac-8c0b-367b5b310899 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.571347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 422/855] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:53.577070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.577655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.578158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.596344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:53.596919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dc13-061c-4fa3-8434-c2aa5b8038df None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:53.598144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:53.642559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:53.643617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.644129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.645357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.646338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:53.647319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:53.649531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:53.657173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 434/856] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:42:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:53.664944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:53.682881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 423/857] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:53.785764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:53.800776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b604ff-6cf2-43ef-a16d-317e5779a937 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:53.836479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:53.837435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:53.837972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:53.839074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:53.839924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:53.840433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:53.842023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:53.847190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 435/858] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:42:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:53.962393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.072768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9b16-3274-4c6d-91ba-38f7436b5388 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.074291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:54.109859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.110498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.110782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.111059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.122337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:54.127174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:54.128092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:54.128908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 424/859] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:42:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:54.134422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.242134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b9d37-bb2e-49d5-8c63-6563aead115b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.279178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.279926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.280186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.280468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.281135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:54.281441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:54.283110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:54.291608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 436/860] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:42:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:54.297145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.404250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95138ea-8d67-47d1-9f15-bc05143c499a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.439748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.440440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.440720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.440999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.452463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:54.453947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-40ad778e-c837-4003-b773-f2bba8c91c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:54.470585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:54.471572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:54.472522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:54.472522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:54.472522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:54.472522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 425/861] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:54.478634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.488416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec703ce-105c-46ea-b594-b2046d23e547 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.488716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec703ce-105c-46ea-b594-b2046d23e547 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.488970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec703ce-105c-46ea-b594-b2046d23e547 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.491750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 426/862] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:54.496367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.496637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.496839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.497340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:54.517330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:54.518584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:54.552066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:54.595977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96e02d-c7a2-410b-9205-261e68e5397d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.601298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 427/863] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:54.636853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.637522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.637867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.638196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.638840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:54.639146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:54.640510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:54.657867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 437/864] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:54.659419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.765624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6be6e5-bb66-4996-ba9f-8e2ccb448976 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.803778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.804420 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.804700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.804974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.813063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:54.814543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:42:54.827129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:54.828060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:54.828984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 428/865] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:42:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:54.834236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.882900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:54.888675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:54.924039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:54.954963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511977-2bcc-46d4-86ce-e4878e74dc2c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:54.995895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:54.996511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:54.996794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:54.997068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:54.997861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:54.998170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:54.999599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.003277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c183d21-2dfe-4ece-85eb-d905ad11128f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.021713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 438/866] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:42:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:55.023291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.042949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.044117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.044598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.045070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.046038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.047928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4899901-cc93-4465-a9eb-aba757ff8a3e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.073971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 429/867] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:42:54 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:42:55.148038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7e5d-121d-42e1-a0d0-a95977681f0f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.182536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:55.196519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.197135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.197412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.198328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.276234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.277618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.363319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1745-bd41-4cef-9fae-46fc8f20b882 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.363843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1745-bd41-4cef-9fae-46fc8f20b882 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.363843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1745-bd41-4cef-9fae-46fc8f20b882 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.365409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 430/868] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:55.370813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.371100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.371375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.389971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:55.391167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:55.392277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 439/869] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 21:42:55.397562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.428491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:55.481225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:55.494305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 431/870] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:55.519372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d2ce8-b272-49a0-8075-8431b27cffce None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.555823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.556681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.557699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.558148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.568674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.570291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:55.586212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:55.587395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:55.588526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:55.588526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:55.588526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:55.590230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 440/871] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:55.595701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.710974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6f29a-d9cf-40cb-bbe7-e5a7267d4d92 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.752929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.754514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.755014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.755460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.756358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:55.756850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.758423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.765789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.777578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 432/872] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:55.782951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.849176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:55.861470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:55.883131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751878d-ea80-4eba-968b-3c81a8972677 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.900175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c4b2-b996-42da-91ab-5f83acb3d707 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:55.919133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.920049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.920338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.920614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.921201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.923037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d38db0e5-2386-4327-b1b0-2e32273784ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:55.932029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:55.933549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:55.934499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 441/873] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:55.935279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793524b-38f0-40b3-b015-2a3fe7379ac7 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.935279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793524b-38f0-40b3-b015-2a3fe7379ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.936049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793524b-38f0-40b3-b015-2a3fe7379ac7 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.936858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 442/874] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:55.941675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793524b-38f0-40b3-b015-2a3fe7379ac7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:55.943069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:55.943704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.943982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.944250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.951615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:55.953058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b5f6294a-b18c-4b1e-bec7-9cf2b65cf15a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:42:55.964870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:55.965825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:42:55.966725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:55.971010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 433/875] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:55.972066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:55.972351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:55.972627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:55.990648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:55.991845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:56.048265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793524b-38f0-40b3-b015-2a3fe7379ac7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:56.065416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 434/876] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:56.066620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:56.092682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:56.093293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:56.093573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:56.093869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:56.094496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:56.094796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:56.096157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:56.106210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 443/877] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:42:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:56.107013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:56.119053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:56.137797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:56.203716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feef2b-77a5-4fc7-8bfd-899cae54a52f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:56.227098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be9167-94c0-4ccc-aa3d-7bac0581d436 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:56.240419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:56.241033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:56.241308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:56.241579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:56.242336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:56.242745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:56.244183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:56.258851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 435/878] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:56.264809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:56.277059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:56.278710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:56.279515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:56.280004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:56.281153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:56.281943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:56.283714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:56.290804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 444/879] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:42:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:56.348157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:56.397019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dd4c8-c4e6-415a-b6f8-d49a699ca9c2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:56.400251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:56.446774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:56.447240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:56.447518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:56.448432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:56.449159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=1222623a682a4d85bd967e96b9277098) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:56.449823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:56.450924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531fbdb-6060-4892-867a-a50710e010ab None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:56.457329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 436/880] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:42:56 2022] GET /identity/v3/role_assignments?user.id=1222623a682a4d85bd967e96b9277098&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:56.466889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:56.505705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb038b66-379e-4c75-8209-77a585ffc752 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:56.548277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:56.549163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:56.549618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:56.550042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:56.550887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:56.552430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa938fc-c776-458a-908a-922ba44aae0c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:56.573946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 445/881] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:42:56 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:42:57.160274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346b591-3b14-46f0-b60a-fc3d70e9a3ed None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:57.167320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346b591-3b14-46f0-b60a-fc3d70e9a3ed None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:57.167320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346b591-3b14-46f0-b60a-fc3d70e9a3ed None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:57.167320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 437/882] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:57.167657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:57.167950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:57.168229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:57.186719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:57.187905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:57.262407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 446/883] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:57.529570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346b591-3b14-46f0-b60a-fc3d70e9a3ed None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:57.639688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346b591-3b14-46f0-b60a-fc3d70e9a3ed None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:57.675529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:57.676545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:57.677008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:57.677460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:57.678266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:57.679927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:57.697190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:57.698930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:42:57.699813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:57.702327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 438/884] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:57.709632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:57.766770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:57.816531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce8fe2-216d-4222-ae04-10935416d0e0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:57.851795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:57.852417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:57.852700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:57.852975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:57.853631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:57.853990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:57.855861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:57.861066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 447/885] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:57.866533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:57.899345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:57.974608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45de29b-3af2-4ff9-8a72-5ee04fd706a7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:58.010522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:58.011418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.011849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.012264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.013046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:58.013504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:58.015144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:58.022176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 439/886] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:42:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:58.028113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:58.131546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eed34-9c72-448b-8153-6c0de1330ae7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:58.161270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:58.173513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:58.174163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.174444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.174720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.175432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] RBAC: Adding query filter params (interface=public, service_id=a5993f9589f94f9688a684fb265aa7cb, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:58.175752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:58.177121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5384951-3821-4d60-b956-eda348acd49f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:58.182860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 448/887] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:42:58 2022] GET /identity/v3/endpoints?service_id=a5993f9589f94f9688a684fb265aa7cb&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:58.188779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:58.290291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:58.298684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c138e-5d38-4f0c-9b64-819676acd793 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:58.334708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:58.335588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.336018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.336442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.337190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:58.338827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496f2e8-3fff-4c28-a063-a1f1266bdd16 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:58.343677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd923c5-b1fd-46d9-bc8e-88ee97fd4162 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.344098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd923c5-b1fd-46d9-bc8e-88ee97fd4162 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.344478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd923c5-b1fd-46d9-bc8e-88ee97fd4162 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.345575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 440/888] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:42:58 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:42:58.346487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 449/889] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:58.357629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc534f4-057f-41dc-811f-3ed722b08790 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.357629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc534f4-057f-41dc-811f-3ed722b08790 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.357629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc534f4-057f-41dc-811f-3ed722b08790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.373505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-acc534f4-057f-41dc-811f-3ed722b08790 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:58.374764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-acc534f4-057f-41dc-811f-3ed722b08790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:58.452698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 441/890] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:58.549036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e6685-9253-423e-9b8a-0c5b3e250f9f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.549386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e6685-9253-423e-9b8a-0c5b3e250f9f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.549582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e6685-9253-423e-9b8a-0c5b3e250f9f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.551217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 450/891] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:42:58.556645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.558056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.558512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.577025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:58.578682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:58.644937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577bb04-6e96-4a0d-9849-8e54bcd2a65f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.645223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577bb04-6e96-4a0d-9849-8e54bcd2a65f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.645489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577bb04-6e96-4a0d-9849-8e54bcd2a65f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.647152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 451/892] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:42:58.657622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.657622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.657622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.658138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 442/893] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:42:58.670675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:42:58.671857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:42:58.752538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 452/894] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:42:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:42:58.766264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:58.788735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:58.875826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17918db-ba89-4294-ae52-7c13a05cc542 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:58.913312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:58.913959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:58.914290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:58.914571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:58.915226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:58.917146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:58.936683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 443/895] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:42:58 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:42:58.938047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.044400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.060935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3af9-fa39-4edd-81ae-b5f25572e7a9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.085966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.108714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.108714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.108714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.109079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.111014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.111014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:59.121644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:59.122571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:42:59.123455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 453/896] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 21:42:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:59.129829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.159084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.186766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906d9c9-2bbc-4a62-95b4-127dd0942051 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.228271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.230032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.230524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.231131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.244555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:42:59.252025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:59.252997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:42:59.253861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:59.255390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 444/897] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:42:59.259157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466aa18-fa19-48bc-a366-8c891133f41f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.261742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.297104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.298135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.298641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.299127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.299978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:59.300494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.302097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.308285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 454/898] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:59.316813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.365964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.378673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2330c-7352-49bf-8438-7adb1414e8d8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.425434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.426514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.426797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.427346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.428028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:59.428592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.430679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.435250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f5a3-e966-4c58-8ef1-85c01e34f0dc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.440968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 445/899] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:59.447499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.450430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.471631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.472245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.472526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.472798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.473434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:59.473769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.475147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.481743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 455/900] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:42:59.488453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.568514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b50a7-698a-4b51-843d-5732ab047910 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.603821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.604464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.604745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.605021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.606126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a2c5-c088-45d9-8288-7cccd5d224c8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.608766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.616318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.617822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:42:59.635095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:42:59.637958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:42:59.640109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:42:59.640109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:42:59.640109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:42:59.648770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 446/901] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:42:59.648770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.649216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e709c-d519-4976-8950-96882565369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.649216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e709c-d519-4976-8950-96882565369c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.649216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e709c-d519-4976-8950-96882565369c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.649216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e709c-d519-4976-8950-96882565369c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.649880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e709c-d519-4976-8950-96882565369c None admin] RBAC: Adding query filter params (interface=public, service_id=3407da3786c94ebc8f98a6ad6723c31a, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:59.649880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e709c-d519-4976-8950-96882565369c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.649880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e709c-d519-4976-8950-96882565369c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.655274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 456/902] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/endpoints?service_id=3407da3786c94ebc8f98a6ad6723c31a&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:59.661173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e709c-d519-4976-8950-96882565369c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.717855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:42:59.777365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8217-2485-495b-8749-e46eafe099fc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.785615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e709c-d519-4976-8950-96882565369c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.820919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.821932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.822360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.823110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.824175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:42:59.824663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.826695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.832186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:42:59.832924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:42:59.833308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:42:59.833716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:42:59.834437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:42:59.836409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:42:59.841884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 447/903] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:42:59.847609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:42:59.847987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 457/904] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:42:59.968756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7eae-1c0c-4b69-b939-aced46dda960 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:42:59.985707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.014100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.016043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.016043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.016043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.024090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.025559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:00.039555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:00.040587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:00.041573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:00.045488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 448/905] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:42:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:00.046907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.083642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.158675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b72a9-940c-450d-ac08-fcd5e8fd4b11 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:00.194886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.195514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.195796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.196072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.196811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:00.197125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.199235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58474971-ff33-48c1-85f3-7c0cb9a73ca8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.221773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 458/906] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:00.224913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.329073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691c60-596c-4292-a850-3b8ddc6e6ec5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:00.358118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef233-d906-49cb-adcd-c9e2d83f63d4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.358306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef233-d906-49cb-adcd-c9e2d83f63d4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.358652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef233-d906-49cb-adcd-c9e2d83f63d4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.360254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 459/907] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:00.367057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.367492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.367912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.368676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.370104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.370604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.371025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.388510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:00.389908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:00.417568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.444833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.446527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.474179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 460/908] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:00.552053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 449/909] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:00.561621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.585988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.664734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad7e20-d546-4d11-840e-a5d9af1a2103 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:00.700775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.701415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.701720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.701997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.712278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.716344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:00.729512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:00.730483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:00.731357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:00.731357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:00.731357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:00.731357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 461/910] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:00.736487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.779025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.831771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.859265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:00.870874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcf0a7-33b3-4d31-b2e1-8f70e4a7cc88 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:00.893676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefd7f9-f48a-4f59-a83d-7325cf06e75c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:00.915311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.917167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.917661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.919732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.920865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:00.921345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.923470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.933794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:00.934630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:00.935011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:00.935409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:00.936100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:00.937607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:00.943994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 450/911] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:00.945744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:00.953609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:00.954666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 21:43:00.955512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:00.957395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 462/912] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:00.961664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.069850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf33f72-9247-4fbe-9fe8-35e868dedc3a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.075363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04de793-fded-495a-8a12-9bd97a0ffbd8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.106625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.107199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.107478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.107763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.109811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.115409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.116987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.117341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.117983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.118263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.118537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.119165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:01.120136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.121852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae3cbf-29a7-4889-96d2-8999bed4cebe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.124029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:01.129971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:01.130853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:01.131742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:01.132997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 463/913] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:01.133704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f17ac-6e45-4f6d-ac8d-551e097cf7e7 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.133992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f17ac-6e45-4f6d-ac8d-551e097cf7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.134269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f17ac-6e45-4f6d-ac8d-551e097cf7e7 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.137644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 464/914] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:01.137773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 451/915] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:01.137851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f17ac-6e45-4f6d-ac8d-551e097cf7e7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.137922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.241692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acfd9-1b41-48d5-846f-cfe7a1b4255e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.258974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f17ac-6e45-4f6d-ac8d-551e097cf7e7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.279564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.280172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.280777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.280777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.281433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:01.281774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.283206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf7497-1baf-4194-999b-0529f178abac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.295254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 452/916] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:01.296081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142299-de27-4d6a-865e-199231982769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.296696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142299-de27-4d6a-865e-199231982769 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.296983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142299-de27-4d6a-865e-199231982769 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.297259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142299-de27-4d6a-865e-199231982769 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.297929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142299-de27-4d6a-865e-199231982769 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:01.298433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142299-de27-4d6a-865e-199231982769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.299365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.299938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.300041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142299-de27-4d6a-865e-199231982769 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.300507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.309936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 465/917] 10.222.0.82 () {56 vars in 1114 bytes} [Thu Aug 18 21:43:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:01.309936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142299-de27-4d6a-865e-199231982769 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.324397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:01.325740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:01.380925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.387042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.400330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 453/918] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:01.401800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.443919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142299-de27-4d6a-865e-199231982769 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.488978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.489666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.490325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.490608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.491346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] RBAC: Adding query filter params (role.id=04d1aad1801248d5a677e6420baee70b, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=1222623a682a4d85bd967e96b9277098) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:01.491659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.493205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.506564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac915-4f40-49db-8184-1a7c41e3561c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.509638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 466/919] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:01 2022] GET /identity/v3/role_assignments?user.id=1222623a682a4d85bd967e96b9277098&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=04d1aad1801248d5a677e6420baee70b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:01.560426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.561059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.561340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.561633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.562324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.563735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.590879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 454/920] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:01.700741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.852691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75521f6-296c-4b14-abab-e4c8177802da None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:01.881068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.899816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:01.900475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.901301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.901578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.902233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:01.903934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a0a6e561-d3cd-41d6-bf56-10b4bf524991 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:01.915266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:01.916352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:01.917766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 467/921] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:01.923245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:01.952266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b6d5a-62b8-4df6-a653-774a404c1bc3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.952562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b6d5a-62b8-4df6-a653-774a404c1bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.952821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b6d5a-62b8-4df6-a653-774a404c1bc3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.954571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 468/922] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:01.959740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:01.960044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:01.960283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:01.979768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:01.986236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:01.987909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:02.079244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ad27-28c9-4e48-b66d-30eb4c5663f3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.125707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 469/923] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:02.155469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.156635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.157237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.158006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.159314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:02.160150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.161923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.174055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 455/924] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:02.176147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.314430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2119fcf-fa65-44dd-8ea0-fbd673ac1a34 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.331896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:02.349166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.349816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.350089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.350359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.350983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:02.351282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.352627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.363147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 470/925] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:02.364532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.455224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.478456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725881b-64b6-48c3-8435-9cd328d7535e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.481737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:02.520221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.521095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.521369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.521660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.522524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] RBAC: Adding query filter params (interface=public, service_id=0cde551eade94d178bb178edc9fbd95d, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:02.522827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.524487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.532024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 456/926] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/endpoints?service_id=0cde551eade94d178bb178edc9fbd95d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:02.538715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.590089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c46a2-7141-4ec3-baa1-eb21673c8a85 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.636318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19560c97-8640-406c-a29b-733014040bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.636963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19560c97-8640-406c-a29b-733014040bdb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.637248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19560c97-8640-406c-a29b-733014040bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.637526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19560c97-8640-406c-a29b-733014040bdb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.638454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19560c97-8640-406c-a29b-733014040bdb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.640741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19560c97-8640-406c-a29b-733014040bdb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-19560c97-8640-406c-a29b-733014040bdb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:02.652124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:02.653125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:02.659395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 471/927] 10.222.0.82 () {56 vars in 1132 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:02.661884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19560c97-8640-406c-a29b-733014040bdb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.692357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfbf02-e937-4e2a-98b4-d92d64ef087a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.715607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:02.742558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.743334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.743706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.744075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.744812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.746311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.754833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 457/928] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/services => generated 1497 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:43:02.779477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19560c97-8640-406c-a29b-733014040bdb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.818156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.818789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.819071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.819344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.820069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:02.820379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.821802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.832737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 472/929] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:02.834321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:02.896564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:02.952421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06192f4c-ff99-4dbf-8ffa-d80c7b5b3c99 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:02.988960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:02.989962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:02.990459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:02.990937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:02.991769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:02.992267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:02.993907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf5f3b-16f2-46c7-9adf-ea6f3b8a289d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:02.999709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 458/930] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:03.007304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:03.116843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b38e-a5d4-4c35-b1cd-acde4e6bb12e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:03.127576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:03.151541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:03.152410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:03.152832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:03.153206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:03.154428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:03.155851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4ce13-4c48-4d20-ac1c-8738cd7b0f8e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:03.185843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 473/931] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:03.578001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6580-c735-41e8-8833-2954dd6fe9f9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:03.578359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6580-c735-41e8-8833-2954dd6fe9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:03.578555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6580-c735-41e8-8833-2954dd6fe9f9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:03.580207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 459/932] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:03.585513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba09f3-c7c1-4031-9653-cab0e6411b8f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:03.585831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba09f3-c7c1-4031-9653-cab0e6411b8f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:03.586106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba09f3-c7c1-4031-9653-cab0e6411b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:03.607673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2aba09f3-c7c1-4031-9653-cab0e6411b8f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:03.609049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2aba09f3-c7c1-4031-9653-cab0e6411b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:03.699205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 474/933] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:03.797516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b28d-ea03-4930-b0ba-c8118baee93b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:03.797876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b28d-ea03-4930-b0ba-c8118baee93b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:03.798151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b28d-ea03-4930-b0ba-c8118baee93b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:03.799743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 460/934] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:03.805356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:03.805663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:03.805940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:03.826010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:03.827191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:03.897694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 475/935] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:03.986986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b28d-ea03-4930-b0ba-c8118baee93b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.087313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.095461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b28d-ea03-4930-b0ba-c8118baee93b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.131759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.132391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.132668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.132939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.144030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:04.148742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:04.149660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.150501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 461/936] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:43:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:04.156315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.183095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.188138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.299109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc08e-3bf6-41c9-82dd-344867890ac7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.312099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbae921-444a-476d-8d3d-b326892044e5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.338004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.339178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.339453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.339748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.340401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:04.340706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.342235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.350658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.351645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.352154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.352632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.353499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.355247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.356489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 476/937] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:04.357464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.358921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING keystone.server.flask.common [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 21:43:04.379362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 21:43:04.379362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 21:43:04.379362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [parameters: {'id': 'a76a7554ee0f487fa1c03671b12e4b18', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 21:43:04.379362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a76a7554ee0f487fa1c03671b12e4b18.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a76a7554ee0f487fa1c03671b12e4b18. Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:43:04.392096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 18 21:43:04.392936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Aug 18 21:43:04.393951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 21:43:04.394864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 21:43:04.395755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:43:04.403161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'id': 'a76a7554ee0f487fa1c03671b12e4b18', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.406289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:04.409116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a76a7554ee0f487fa1c03671b12e4b18. Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 462/938] 10.222.0.82 () {60 vars in 1140 bytes} [Thu Aug 18 21:43:04 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:04.411358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.427119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.475469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.497962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a53a-7caf-43fa-967d-d17893d8cdf4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.532508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f26e5d-c2c3-4467-b34e-980848ece3e6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.544562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.546090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.546608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.547099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.561447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.563160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.576711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.577699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.578179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.578740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:04.580443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:04.581348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:04.582278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:04.582278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:04.582278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:04.585948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 477/939] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:04.587729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.590306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.593551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:04.609830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:04.610741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:04.611677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:04.619991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 463/940] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:04.620856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.699850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.727527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bbed5-41a8-46d4-a178-f36c29a875f4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.734817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cb5d-a232-4d9b-8aa6-263f88428c48 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.767321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.768307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.768900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.769375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.770314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:04.770820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.772519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.776370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:04.780213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.780986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.781324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.781663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.782339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:04.782641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:04.785407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78f865-e613-45ec-a743-cf26824ff2c2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:04.789871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 478/941] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:04.797243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:04.821629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 464/942] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:04.845629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ea9e-f948-4f7b-a7f3-fff004dd165d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.845629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ea9e-f948-4f7b-a7f3-fff004dd165d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.845629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ea9e-f948-4f7b-a7f3-fff004dd165d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.845629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 465/943] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:04.851686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.851969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.852246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:04.874592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:04.877324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:04.945684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df59a5e-7a91-41c1-8b6e-3a5efe70e9e7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:04.967409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 466/944] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:04.997482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:04.998164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:04.998443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:04.998715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.007411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.008857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:05.022332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:05.023301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:05.024227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 479/945] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:43:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:05.030225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.096705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.137734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d11577-2558-4704-9c57-f1241a3281fe None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.173533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.174435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.174923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.175243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.176005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:05.176314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.177844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700ec8f-9f43-4a87-9e67-cd9e72e27369 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.196242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 467/946] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:05.202930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.240756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349419dd-200e-48c2-a34f-6602a1acacc5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.241195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349419dd-200e-48c2-a34f-6602a1acacc5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.241320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349419dd-200e-48c2-a34f-6602a1acacc5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.243056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 468/947] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:05.249571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.249870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.250139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.269143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:05.270358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:05.271213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.320701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e687fb-675e-47fb-806d-a15836e3f5eb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.346123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 469/948] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:05.346123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.360906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.361541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.361846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.362119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.376508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.379151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:05.401077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:05.402366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:05.403790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:05.403790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:05.403790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:05.406316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 480/949] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:05.408250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.417628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.476136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cda68-4afa-4663-97e2-f6e6537ebaff None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.520409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.521076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.521446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.521815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.522471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.524111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.533786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebf83b-2e0d-4032-8e3f-d9955f4cd115 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:05.534404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:05.535378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.536209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 470/950] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:05.542054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.544055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.575634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081393f-f897-454d-a969-348f870d3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.576544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081393f-f897-454d-a969-348f870d3305 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.576828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081393f-f897-454d-a969-348f870d3305 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.577100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081393f-f897-454d-a969-348f870d3305 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.577765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081393f-f897-454d-a969-348f870d3305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:05.578074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081393f-f897-454d-a969-348f870d3305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.579929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081393f-f897-454d-a969-348f870d3305 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.599571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 481/951] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:05.601505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081393f-f897-454d-a969-348f870d3305 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.662925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844317d7-2d71-4674-ba90-6626ebeda49c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.676693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.702396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.703008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.703285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.703555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.704181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:05.704482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.705888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.715110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 471/952] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:05.715110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.725273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081393f-f897-454d-a969-348f870d3305 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.762973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.763769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.764046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.764706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.773081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.775115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:05.789907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:05.790805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 482/953] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:05.797252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.817827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d76119-ef5b-417b-a0cb-1ba20dd66ce9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.831306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.853071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.854253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.854537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.854809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.855396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.856766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:05.865514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:05.866705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:05.867590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 472/954] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:05.872911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.913203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee54e9-7acd-40d1-8d35-b537b09b757e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:05.919630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:05.950761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:05.951379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:05.951659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:05.951933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:05.952560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:05.952866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:05.954266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:05.959267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 483/955] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:05.965217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:05.982042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a81ba1-0ffe-4efa-bcde-13813a558bd2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:06.020366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:06.020979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.021254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.021522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.022330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:06.022650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:06.024245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:06.037324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 473/956] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:06.038144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:06.065866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:06.076456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08b80b-1cb9-4c8c-bd50-842d49919fe0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:06.110192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:06.110804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.111077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.111344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.111962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:06.112259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:06.113622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:06.123420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 484/957] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:06.123420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:06.148438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4274c-aeca-468f-a39f-9b3de04a603c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:06.149998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:06.185558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:06.186204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.186481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.186749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.187407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:06.188775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:06.217639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 474/958] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:06.217639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:06.233810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392e11-f1d0-4197-b988-d1faddcaab33 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:06.280054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:06.280754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.281168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.281933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/1222623a682a4d85bd967e96b9277098/roles/04d1aad1801248d5a677e6420baee70b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.285331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=1222623a682a4d85bd967e96b9277098, role_id=04d1aad1801248d5a677e6420baee70b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:06.287729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc3312-4e54-4613-8214-af06e4fab6de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:06.309271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 485/959] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:43:06 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/1222623a682a4d85bd967e96b9277098/roles/04d1aad1801248d5a677e6420baee70b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:43:06.356168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086477-cdee-4a3e-9b5a-4836c3aebf94 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:06.400264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:06.400958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.401236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.401509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.402216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:06.402572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:06.404230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3ad20-0cea-418f-8c4b-a13f45d2a977 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:06.409648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 475/960] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:06.921108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef1e0-1bc5-4876-8654-87465c74f574 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.921488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef1e0-1bc5-4876-8654-87465c74f574 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.921695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef1e0-1bc5-4876-8654-87465c74f574 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.923290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 486/961] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:06.928752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:06.929039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:06.929314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:06.949193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:06.950749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:06.968880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:07.023808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 476/962] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:07.290624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ef1e0-1bc5-4876-8654-87465c74f574 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:07.307883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:07.406986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ef1e0-1bc5-4876-8654-87465c74f574 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:07.443010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:07.443646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:07.443929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:07.444207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:07.444869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:07.446377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:07.449101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING keystone.server.flask.common [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 21:43:07.464110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.sql.core [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 21:43:07.464110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 21:43:07.464110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [parameters: {'user_id': 'c4c9f04a1fe9475d92486a811752279a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 21:43:07.464110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81458) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Aug 18 21:43:07.482575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:43:07.483523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application yield resource Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise self.value Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application session.commit() Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 21:43:07.484734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.flush() Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application compat.raise_( Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 21:43:07.485714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 21:43:07.486619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 21:43:07.487483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c4c9f04a1fe9475d92486a811752279a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:07.488332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 18 21:43:07.489155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:07.492944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 487/963] 10.222.0.82 () {60 vars in 1131 bytes} [Thu Aug 18 21:43:07 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:07.500529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:07.604453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84e2ee-473b-4590-ab94-43d2667a83f4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:07.642978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:07.643611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:07.643886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:07.644156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:07.651355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:07.652796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-af9b2da7-c274-4b0b-800b-bc834f39b88d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:07.664534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:07.665518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:07.666979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 477/964] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:07.674107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:07.765896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:07.802096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80cd0-6db1-472e-8c7e-a5e31fee8e88 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:07.837269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:07.838931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:07.839419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:07.839897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:07.840816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:07.841331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:07.843298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9e2b9-53a4-46f8-ab6b-8b3771e1282d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:07.883857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 488/965] 10.222.0.82 () {56 vars in 1101 bytes} [Thu Aug 18 21:43:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:08.262893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff215-51d7-41c9-9408-dfeccd83b816 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.263241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff215-51d7-41c9-9408-dfeccd83b816 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.263370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff215-51d7-41c9-9408-dfeccd83b816 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.264873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 478/966] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:08.270208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264fdc-7fca-401e-862c-83bb734fa5cb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.270498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264fdc-7fca-401e-862c-83bb734fa5cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.270776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264fdc-7fca-401e-862c-83bb734fa5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.290358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e5264fdc-7fca-401e-862c-83bb734fa5cb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:08.292446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e5264fdc-7fca-401e-862c-83bb734fa5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:08.342166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613d449-e029-4625-9b18-31b511ada116 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.342480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613d449-e029-4625-9b18-31b511ada116 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.342737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613d449-e029-4625-9b18-31b511ada116 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.344216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 479/967] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:08.349047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce810-3da1-4bb3-a48d-a990d0dfa661 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.349337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce810-3da1-4bb3-a48d-a990d0dfa661 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.349631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce810-3da1-4bb3-a48d-a990d0dfa661 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.350861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 480/968] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:08.351864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.352141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.352416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.364936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 489/969] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:08.366210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.366490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.366785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.371217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:08.372406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:08.380070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:08.386089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:08.387274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:08.444965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 481/970] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:08.448762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:08.459897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 490/971] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:08.636470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:08.714587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:08.747508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e6ba6-7618-4533-8ea4-407d7b3b5437 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:08.782544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:08.783162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.783440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.783709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.784290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:08.785691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:08.794191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:08.795161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:08.795998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 482/972] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:08.796817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:08.806643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:08.813728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cfa37-0781-4976-a456-7b8ccf9c6db1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:08.847434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:08.848065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.848343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.848619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.858340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:08.863074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:08.864217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:08.865079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 491/973] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:43:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:08.865841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:08.898954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f7d1-e3fe-405b-9b2a-87a25d6cd7df None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:08.918297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:08.933949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:08.934547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:08.934821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:08.935090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:08.935724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:08.937113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:08.951377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 483/974] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:43:08 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:08.952168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:08.973923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4619-2f6a-43f6-aa8b-71ce2167cd10 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.022812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.023439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.023718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.023992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.024631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.024937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.026386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.032239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 492/975] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:09.038103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.040168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.080034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef66b0-446a-4058-86c6-6224d52539a6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.121502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.122160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.122438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.123358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.123358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.123974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.125041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.134113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 484/976] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:09.139563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.156376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a850e6-a003-4382-8912-4742c2de8c15 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.171596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.195013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.195634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.195913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.196189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.196822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.197129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.198556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.203562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 493/977] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:09.209275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.255052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c9f3c-606d-4e1a-a052-d9bda6b60fa4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.282742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.292145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.292868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.293153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.293422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.304489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.305965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:09.320385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:09.321412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:09.322569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:09.322569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:09.322569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:09.322753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389f5ec-e658-4cb8-bf18-6a4d0cb0bf52 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.326480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 485/978] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:09.327148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.363650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.364271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.364572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.364829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.365540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] RBAC: Adding query filter params (interface=admin, service_id=0cde551eade94d178bb178edc9fbd95d, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.365876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.367237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.373634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 494/979] 10.222.0.82 () {56 vars in 1249 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/endpoints?service_id=0cde551eade94d178bb178edc9fbd95d&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:09.379199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.430702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.441900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc260-18e3-43f6-863d-055fd83572ba None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.481342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.481979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.482259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.482531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.483174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.483479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.484855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.492514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03133d-3ecb-47c5-9860-002ca0adb643 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.498528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 486/980] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:09.504026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.532116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.532724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.533223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.533392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.534028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.535381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.541622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 495/981] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:43:09.547900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.613968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6c142-1fa8-489d-9e5c-1add19b8a54b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.652289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.652937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.653290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.653751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.661723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.663211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:09.675618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:09.676710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 21:43:09.678132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:09.681547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 487/982] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:09.682786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.791222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f4f7-2122-4c74-8249-f55371a04466 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:09.793536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.828454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:09.829094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:09.829513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:09.829925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:09.830659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:09.830964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:09.832368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:09.853630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 496/983] 10.222.0.82 () {56 vars in 1149 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:09.856835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:09.974270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:09.978933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72983298-c3be-4c5f-bbcf-3e97428a0bce None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:10.016200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:10.016812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.017087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.017355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.087139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:10.089284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:10.183290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:10.188163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 488/984] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:09 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:10.193784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:10.305189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2026ec-4634-4b2a-a9bc-f7ed944882cf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:10.340338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:10.340962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.341226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.341494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.351333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:10.352749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:10.366988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:10.367915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:10.368797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:10.368797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:10.368797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:10.369900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 497/985] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:10.373817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:10.477002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be257e-bb74-462a-b03b-e9563671b729 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:10.512464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:10.513076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.513350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.513638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.514278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:10.514578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:10.515938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:10.528859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 489/986] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:10.534658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:10.550894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:10.640212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfce49-97f8-4856-8b9f-eb9b77059804 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:10.675951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:10.676584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.676867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.677140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.684466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:10.685925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:10.699643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:10.700593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:10.706561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 498/987] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:10.708471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:10.774045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:10.817313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9232-171a-4207-8b03-a8ac57dc91de None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:10.850747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:10.851353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.851628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.851895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.852579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:10.852883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:10.854978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62146-4635-4382-8744-fc20d54e2f18 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:10.872314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 490/988] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:10.873935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:10.908801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7a94-b55c-4f1e-96ed-aca26b8b8d3d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.909003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7a94-b55c-4f1e-96ed-aca26b8b8d3d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.909272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe7a94-b55c-4f1e-96ed-aca26b8b8d3d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.910802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 491/989] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:10.915821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:10.916233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:10.916935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:10.938268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:10.939459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:10.956562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:10.982553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e848d-62a5-4616-a83e-57f94003912f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.016893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 492/990] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:11.017969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:11.018582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.018863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.019134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.019860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] RBAC: Adding query filter params (role.id=04d1aad1801248d5a677e6420baee70b, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=1222623a682a4d85bd967e96b9277098) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:11.020167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:11.021588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:11.028679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 499/991] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:10 2022] GET /identity/v3/role_assignments?user.id=1222623a682a4d85bd967e96b9277098&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=04d1aad1801248d5a677e6420baee70b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:11.291015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.341238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:11.395002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f21ea-6700-499a-ba37-0e0541fb99c2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.430947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:11.431606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.431917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.432187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.432772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:11.434192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cb1929bb-7526-4f2a-b465-1841f1719f51 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:11.443075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:11.444065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:11.445477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 493/992] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:11.453731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.495619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfb0a-8f4c-441b-bb50-7a0c820017b6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.495966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfb0a-8f4c-441b-bb50-7a0c820017b6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.496165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfb0a-8f4c-441b-bb50-7a0c820017b6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.497505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 494/993] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:11.502608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.502885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.503156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.522430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:11.523619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:11.572570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281b35-45c2-4089-9ae9-889ac0688eb2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.595297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 495/994] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:11.612364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:11.612982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.613265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.613538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.614206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:11.614511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:11.615911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:11.626198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 500/995] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:11.627888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.680892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:11.741747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272ce8-df2c-4efb-b16c-45ffdef0c060 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.776443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:11.777054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.777333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.777617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.778254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:11.778810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:11.780179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:11.785557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 496/996] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:11.791824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.794832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:11.866114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.901254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f43d0-c42a-424c-a57a-c8f4610f9024 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.939256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:11.939871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:11.940152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:11.940425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:11.941125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] RBAC: Adding query filter params (interface=public, service_id=982b881031b24b44812279c9804e1061, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:11.941435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:11.942831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:11.948206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 501/997] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/endpoints?service_id=982b881031b24b44812279c9804e1061&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:11.953475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:11.975890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cefd75-bdf1-45b3-8ea9-8c3e159020d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:11.985418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:12.015876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:12.016514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:12.016791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:12.017059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:12.017680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:12.019047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:12.027293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:12.030143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:12.031148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:12.031981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 497/998] 10.222.0.82 () {56 vars in 1126 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:12.037816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:12.075221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109588c-4caf-472b-a150-e6246501f926 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:12.112211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:12.112826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:12.113107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:12.113375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:12.114126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:12.115362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:12.129719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 502/999] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:43:11 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:43:12.156497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3c8d-3451-4c17-8c9c-df8a95b523ac None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:12.197360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:12.198021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:12.198303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:12.198576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:12.199219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:12.199525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:12.200931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:12.206686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 498/1000] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:12.212371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:12.323871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a58a42-e8e9-46b5-8b98-65a169e49324 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:12.351901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:12.359542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:12.360344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:12.360628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:12.360903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:12.361540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:12.361875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:12.363241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f7ba-e1c6-4961-a308-e56920b47a5c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:12.368258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 503/1001] 10.222.0.82 () {56 vars in 1122 bytes} [Thu Aug 18 21:43:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:12.374375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:12.413439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:12.481827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1675-8bd8-4595-bd73-8414e4387a12 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:12.517067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:12.517705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:12.517987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:12.518259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:12.518959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:12.520334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4142c-d741-425b-a41d-c9cb8852d9ce None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:12.537444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 499/1002] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:13.031483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9fddc-f53f-40d3-955a-79c8f5e17b23 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:13.031980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9fddc-f53f-40d3-955a-79c8f5e17b23 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:13.032342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9fddc-f53f-40d3-955a-79c8f5e17b23 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:13.033731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 504/1003] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:13.038765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:13.039213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:13.039289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:13.057696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:13.058866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:13.133657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 500/1004] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:13.399853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9fddc-f53f-40d3-955a-79c8f5e17b23 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:13.507898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9fddc-f53f-40d3-955a-79c8f5e17b23 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:13.542509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:13.543142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:13.543421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:13.543688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] PATH_INFO: `/v3/services/image` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:13.544252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:13.545759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:13.554331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:13.555313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:13.556182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:13.564320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 505/1005] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 21:43:13 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:13.564531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:13.587287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:13.667626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdf290-620b-4a4f-91b9-c1dfa08cd82c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:13.707232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:13.707856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:13.708134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:13.708405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:13.709040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] RBAC: Adding query filter params (name=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:13.709342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:13.710740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae367891-61c5-4a49-bdbb-3ac9cc4cbe83 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:13.718776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 501/1006] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:13.724260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:13.786102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:13.829530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfed7c-15a6-4a59-a158-7e4059821510 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:13.863962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:13.864571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:13.864850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:13.865117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:13.865768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] RBAC: Adding query filter params (type=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:13.866073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:13.867439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342be20b-a874-4660-8893-04883ceaab0e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:13.872036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 506/1007] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:14.057771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1500-5903-4fd5-9825-c918ecfb5682 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.058291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1500-5903-4fd5-9825-c918ecfb5682 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.058367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1500-5903-4fd5-9825-c918ecfb5682 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.059811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 502/1008] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:14.064924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce54b-5f66-4e15-845a-c7c9384bfbbf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.065339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce54b-5f66-4e15-845a-c7c9384bfbbf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.065517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ce54b-5f66-4e15-845a-c7c9384bfbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.084147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5f4ce54b-5f66-4e15-845a-c7c9384bfbbf None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:14.085352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5f4ce54b-5f66-4e15-845a-c7c9384bfbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:14.157640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 507/1009] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:14.426926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac1500-5903-4fd5-9825-c918ecfb5682 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:14.468397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36783e6-6a6c-46dc-ae35-d9e8a4ec9b54 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.468720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36783e6-6a6c-46dc-ae35-d9e8a4ec9b54 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.468960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36783e6-6a6c-46dc-ae35-d9e8a4ec9b54 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.470339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 508/1010] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:14.475564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.475846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.476122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.494089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:14.495875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:14.519397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:14.537762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac1500-5903-4fd5-9825-c918ecfb5682 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:14.545942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:14.573640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 509/1011] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:14.579191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:14.579828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.580103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.580373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.580955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:14.582367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:14.591100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:14.592150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 21:43:14.593016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:14.597179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 503/1012] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:14.598506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:14.703236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed58a3b-112c-4379-8ed3-b52fc27e5c91 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:14.742667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:14.743285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.743569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.743845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.745028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:14.745340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:14.746732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:14.756330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 510/1013] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:14.757628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:14.837317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:14.864037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb6ce5-52e0-475a-8f1b-4bd8da279f36 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:14.884325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:14.902086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:14.902701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.902981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.903251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.904142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:14.904478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:14.905874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:14.910802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 504/1014] 10.222.0.82 () {56 vars in 1112 bytes} [Thu Aug 18 21:43:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:14.916300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:14.934969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:14.950755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306e9e9-79e7-4c9a-ae48-4391ee7c6ceb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:14.988256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:14.989222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:14.989742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:14.990258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:14.991436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:14.992862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:14.994632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 511/1015] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:15.000415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:15.027259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69307c-a5c9-4240-a31e-7d73a1ae7dfc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:15.062730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:15.063342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:15.063601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:15.063870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:15.064544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:15.066011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ee017-12c5-4cd4-8694-b6f5f6aeebca None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:15.089204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 505/1016] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:15.110745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3d7d-76f1-4b26-b937-347a6ee5564b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:15.127374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:15.158158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:15.158781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:15.159061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:15.159339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:15.159997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:15.161483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbb20d-270d-427b-b6b7-6d1aeb2efe19 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:15.198035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 512/1017] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:43:14 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:43:15.776903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46174519-cec0-4ea6-93a3-308cc61b756f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:15.777235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46174519-cec0-4ea6-93a3-308cc61b756f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:15.777451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46174519-cec0-4ea6-93a3-308cc61b756f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:15.779152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 506/1018] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:15.784392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ab8-14a6-4d06-b619-beeef036e782 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:15.784690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ab8-14a6-4d06-b619-beeef036e782 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:15.784974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ab8-14a6-4d06-b619-beeef036e782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:15.804156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-142c9ab8-14a6-4d06-b619-beeef036e782 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:15.805357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-142c9ab8-14a6-4d06-b619-beeef036e782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:15.873448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 513/1019] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:15 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:16.140673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46174519-cec0-4ea6-93a3-308cc61b756f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:16.219523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:16.263170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46174519-cec0-4ea6-93a3-308cc61b756f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:16.302109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:16.302722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:16.302996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:16.303323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:16.303972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:16.305359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c9518-df81-4a2a-a251-b08931ff5799 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:16.321854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 507/1020] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:43:16 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:17.096845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fd9d-3218-46b3-b687-3293a067c04c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.097392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fd9d-3218-46b3-b687-3293a067c04c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.097392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2fd9d-3218-46b3-b687-3293a067c04c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.098770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 514/1021] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:17.104456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1a77-0e13-4a8e-a055-c43822d376f5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.104743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1a77-0e13-4a8e-a055-c43822d376f5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.105020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1a77-0e13-4a8e-a055-c43822d376f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.123405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-685f1a77-0e13-4a8e-a055-c43822d376f5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:17.124587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-685f1a77-0e13-4a8e-a055-c43822d376f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:17.201679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 508/1022] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:17.230811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25038f-3f68-42a3-8c4a-2570b6f6703e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.231108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25038f-3f68-42a3-8c4a-2570b6f6703e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.231333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25038f-3f68-42a3-8c4a-2570b6f6703e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.232816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 515/1023] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:17.237861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.238150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.238426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.257374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:17.258607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:17.311566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:17.330050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 509/1024] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:17.465716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25038f-3f68-42a3-8c4a-2570b6f6703e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:17.555763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:17.566672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25038f-3f68-42a3-8c4a-2570b6f6703e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:17.598221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:17.601789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:17.602427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.602708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.602979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.618729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:17.625405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:17.626384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:17.627240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:17.630089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 516/1025] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:43:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:17.632728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:17.706850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777112-ffc5-4a94-ad29-4e5c05e3e144 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:17.744212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d98f7-e035-46ce-9bd7-364142297f64 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:17.748668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:17.749318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.749614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.749925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.751042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:17.752499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:17.754269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 510/1026] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:17.760201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:17.782363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:17.782831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.783110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.783389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.784056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:17.784362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:17.785757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:17.794720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 517/1027] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:17.800174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:17.817279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:17.823316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:17.874004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914360b-8d77-4a2d-9fa6-e24ca43929ec None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:17.909298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559184-61f9-4955-b680-856e66ea0466 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:17.909743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:17.910357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.910638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.910939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.911564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:17.913003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:17.945624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 511/1028] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:43:17 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:43:17.946114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:17.946733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:17.947873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:17.948189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:17.960829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:17.962427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:17.980996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:17.982214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:17.983288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:17.983288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:17.983288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:17.984018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 518/1029] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:17.988878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.101714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cedfd-36fc-4c1f-a64a-3b4c50923f1c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.115658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.148592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.149552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.149966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.150370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.151418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:18.151880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.154309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45643a85-2210-439a-ae85-7454d9e8b4b7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.171033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 512/1030] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:18.176619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.247708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.284776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63066c5-319c-44de-a269-f7dc31e54d50 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.296459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73471c-90b2-4c72-ba14-b7dc4a410558 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.296960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73471c-90b2-4c72-ba14-b7dc4a410558 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.297047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73471c-90b2-4c72-ba14-b7dc4a410558 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.298666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 513/1031] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:18.303483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.303767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.304043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.319869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.320507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.320787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.321061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.325710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:18.326961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:18.330519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.331963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:18.343423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:18.344391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:18.345330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 519/1032] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:18.351262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.404775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 514/1033] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:18.456977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0680055-50f2-43c5-a4d2-bf83343359ac None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.484203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.494831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.495743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.496030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.496306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.497133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:18.497454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.499032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.522908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 520/1034] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:18.524112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.624427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.628438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afb235-2c53-4f35-abb6-2cf2ccabdafe None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.666817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.667430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.667709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.667980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.671641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.747804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.749548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.780336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562504c6-3db7-4de4-8cb4-3483bdfa17a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.821253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.821996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.822191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.822488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] PATH_INFO: `/v3/services/image` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.823067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.824437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:18.834959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:18.836001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:18.838828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 521/1035] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:18.843165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.859607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.869506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 515/1036] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:18.870665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:18.895837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:18.949968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061004c-6331-4dac-8c6a-47b656a9d347 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.981324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d042d-8851-44bc-8313-dd685bedb300 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:18.989109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:18.989759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:18.990043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:18.990318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:18.990955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] RBAC: Adding query filter params (name=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:18.991260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:18.992629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:18.997933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 522/1037] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:19.004726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.022184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.022821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.023098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.023361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.035231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.036873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:19.052844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:19.053871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:19.055377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:19.055377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:19.055377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:19.055377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 516/1038] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:19.059745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.119286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8dd9-cda1-49c4-8f87-f4b96a67d938 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.121510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:19.136413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:19.161100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.161695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.161979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.162257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.162885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] RBAC: Adding query filter params (type=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:19.163188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.164558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.172437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 523/1039] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:19.178021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.189064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4511-f420-4dc1-b516-0be211c93039 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.225206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.225849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.226224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.226498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.227162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:19.227542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.228955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.245737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 517/1040] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:19.247836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.288619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95001ed-d8ae-4700-9095-dbbd4319c16a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.324830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.325450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.325766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.326390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.326744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] RBAC: Adding query filter params (interface=public, service_id=aebf3b49fdf247358e74adb07703c463, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:19.327056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.328490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.333797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 524/1041] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/endpoints?service_id=aebf3b49fdf247358e74adb07703c463&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:19.339756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.361954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f298d-ef09-4ab5-b7c8-e882636a41ad None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.377411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:19.387252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:19.400096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.401372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.401680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.402027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.409390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.410949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:19.422116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:19.422995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:19.423896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:19.425915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 518/1042] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:19.429228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.460560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c517b-46e2-4006-9198-6822dcf1a5ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.499456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.500159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.500442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.500716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.501353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.502760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.509802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 525/1043] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:43:19.538269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033def-5d19-4782-9541-b64bb87976a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.577552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.578491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.579031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.579522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.580441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:19.580930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.582604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea28bdb-8942-4055-ba8d-4104905fd2cd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.597398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 519/1044] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:19.604495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:19.713640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2140f04-4b64-4c4b-93af-05d88cd07e98 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:19.721955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:19.751120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:19.751727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.752001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.752271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.752990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] RBAC: Adding query filter params (role.id=04d1aad1801248d5a677e6420baee70b, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:19.753297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:19.754750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06251e4-ee20-4c99-93ae-e0ad90c71fb7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:19.761062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 526/1045] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:19 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=04d1aad1801248d5a677e6420baee70b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:19.942277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bcb71-6c9d-4789-b017-8e99961d861c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.942465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bcb71-6c9d-4789-b017-8e99961d861c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.942749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bcb71-6c9d-4789-b017-8e99961d861c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.944119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 520/1046] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:19.949988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:19.950282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:19.950565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:19.969420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:19.970676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:20.044978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 527/1047] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:20.311438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bcb71-6c9d-4789-b017-8e99961d861c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:20.343073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:20.417323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bcb71-6c9d-4789-b017-8e99961d861c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:20.451979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:20.452613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:20.452893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:20.453166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:20.463249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:20.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:20.468926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:20.469796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:20.471037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 521/1048] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:20.477168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:20.574490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:20.579038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b62c-da7c-4982-878b-77c83e0c3453 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:20.616349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:20.617300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:20.617829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:20.618323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:20.619735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:20.620243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:20.621816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:20.634913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 528/1049] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:20.636794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:20.743021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:20.753100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc6888-114f-4c31-ad30-91d41ed24287 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:20.788053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:20.788686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:20.788966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:20.789237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] PATH_INFO: `/v3/users/cfed0bd9414d41fb947727074728cd5a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:20.803234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] RBAC: Authorizing `identity:get_user(user_id=cfed0bd9414d41fb947727074728cd5a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:20.804690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:20.811292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 522/1050] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:20 2022] GET /identity/v3/users/cfed0bd9414d41fb947727074728cd5a => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:20.813110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:20.916710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814316-b95e-41f4-a01d-e3c7c681c0db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:20.950947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:20.951549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:20.951827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:20.952096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:20.979996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:21.021691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:21.023133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:21.125627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 529/1051] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:21.131224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:21.233102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678b00-29dd-4083-8f64-fe943657fa84 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:21.267704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:21.268333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.268605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.268870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.270005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:21.271433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce5153-6ec9-4a01-87dd-96e6fdd8518e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:21.273204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 523/1052] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:21 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:21.279966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:21.339225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:21.380907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574325bb-2ecc-4641-b068-3534f705eaea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:21.414823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:21.415468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.415702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.416089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.417296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=22756dc870b34229a4747c67aa49a9ae, user.id=cfed0bd9414d41fb947727074728cd5a) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:21.417626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:21.419042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b337-9571-4982-90d8-4d9841f224ae None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:21.433632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 530/1053] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:21 2022] GET /identity/v3/role_assignments?user.id=cfed0bd9414d41fb947727074728cd5a&scope.project.id=22756dc870b34229a4747c67aa49a9ae&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:21.433835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d5025-6338-4147-8852-5c9b8c2bc810 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.433835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d5025-6338-4147-8852-5c9b8c2bc810 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.434397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d5025-6338-4147-8852-5c9b8c2bc810 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.435271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 524/1054] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:21.441219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcffb7a-4866-40fb-9e8c-6f286e859985 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.441574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcffb7a-4866-40fb-9e8c-6f286e859985 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.441795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcffb7a-4866-40fb-9e8c-6f286e859985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.461647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2fcffb7a-4866-40fb-9e8c-6f286e859985 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:21.463028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2fcffb7a-4866-40fb-9e8c-6f286e859985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:21.535307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 531/1055] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:21.658895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3215d-6243-4b93-a370-28cc8b71aad2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.659227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3215d-6243-4b93-a370-28cc8b71aad2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.659464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3215d-6243-4b93-a370-28cc8b71aad2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.660947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 525/1056] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:21.666513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.666813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.667090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.685514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:21.686732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:21.743826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:21.763228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 532/1057] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:21.803470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3215d-6243-4b93-a370-28cc8b71aad2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:21.823947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:21.909567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3215d-6243-4b93-a370-28cc8b71aad2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:21.944465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:21.945091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:21.945368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:21.945661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] PATH_INFO: `/v3/services/image` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:21.946237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:21.947667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:21.956536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:21.957567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 21:43:21.958441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:21.961924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 526/1058] 10.222.0.82 () {56 vars in 1105 bytes} [Thu Aug 18 21:43:21 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:21.964417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.033849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.069301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e64b6-4913-45fe-98da-65d30fe2a4dd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.104540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.105146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.105423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.105724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.106351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:22.106648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.107997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.117805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 533/1059] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:22.118521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.119089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:22.138708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd2d7b-bf06-4f7a-a316-36cdc3ccc84c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.169852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:22.175965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.176700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.176994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.177263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.188726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:22.193397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:22.194324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:22.195166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 527/1060] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:22.200918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.226535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbe764-ab0e-492b-8d72-c099ab0636b9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.263362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.263979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.264263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.264542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.265172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] RBAC: Adding query filter params (type=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:22.265478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.266879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.271916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 534/1061] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:22.277501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.314034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa8b-07b1-48ed-a3cf-ca0aa70e5e59 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.350222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.351041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.351463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.351996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.352665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:22.352969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.355048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.366023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 528/1062] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:22.369293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.373823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:22.381702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d18e-5766-4f6c-a60e-611be8bd8832 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.396146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:22.420289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.420897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.421173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.421440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.422140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.423497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.446248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 535/1063] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:43:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:43:22.487125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c98602-cd1b-4d77-840b-1e7be6d1dc3b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.533356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.534032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.534311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.534583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.546977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.548460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:22.564507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:22.565535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:22.566943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:22.566943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:22.566943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:22.566943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 529/1064] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:22.573729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.679571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90150e4a-8514-4952-b1b1-738437e4ab15 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.720925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.721543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.721850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.722117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.722755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:22.723054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.724413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.736988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 536/1065] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:22.742444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.798391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:22.847093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9bffd-4f82-45a2-9360-56140069d89b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:22.881132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:22.881877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:22.882223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:22.882560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:22.890347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:22.891977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:22.904100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:22.905256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:22.906461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:22.907693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 530/1066] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:22.913180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:22.941832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:23.019949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266a83-847a-41b2-8eca-4f550e40c371 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.057566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.058198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.058475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.058748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.059479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:23.059786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.061199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.083679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 537/1067] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:43:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:23.085066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.186639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9083ee-f8da-4952-9dae-563077c5441b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.200210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:23.223110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.223818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.224165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.224511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.235496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.237005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c4ed11ee-9374-4b23-9697-8586964a6472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:23.252413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:23.253618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:23.254733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:23.254733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:23.254733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:23.255484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 531/1068] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:23.261131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.342223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a4e3-6a20-4076-9156-702bc193386d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.342520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a4e3-6a20-4076-9156-702bc193386d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.342800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a4e3-6a20-4076-9156-702bc193386d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.344152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 532/1069] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:23.349110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.349243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:23.349391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.349720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.366330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7152954-084f-4164-8a6f-a74b3be82a5b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.368865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:23.370126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:23.403239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.404401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.404869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.405209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.405937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:23.406241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.407608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.424768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 538/1070] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:23.425472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.439676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 533/1071] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:23.530859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13132a-f2a5-4f7d-92d6-025447c38972 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.564631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.565260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.565540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.565840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.573537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.575127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:23.587823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:23.588809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:23.590195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:23.595794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 539/1072] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:23.595937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.692903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:23.701221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41487d00-1518-4934-8581-183ea28ab6a8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.707133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.719809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:23.742408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.743034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.743304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.743575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.744272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:23.744574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.746039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.762778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 534/1073] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:23.763890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.821575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efb0b0-fd7b-421b-a450-677a2e6fdfdb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.859172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.859806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.860084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.860624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.872918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:23.877727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:23.878633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:23.879550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 540/1074] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:23.883950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0d29-b773-4508-b257-b0c9889e1b95 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:23.884638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:23.923391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:23.924054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:23.924329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:23.924601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/04d1aad1801248d5a677e6420baee70b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:23.926901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=d441753da1af4f3c8f3b5fb4088d201b, role_id=04d1aad1801248d5a677e6420baee70b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:23.928586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:23.945758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 535/1075] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:43:23 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/d441753da1af4f3c8f3b5fb4088d201b/roles/04d1aad1801248d5a677e6420baee70b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:43:23.962018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:24.003063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94787b-5743-4662-99cd-401caaf053a8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:24.041453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:24.042118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.042396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.042672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.043339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:24.043640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:24.045036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:24.060140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 541/1076] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:24.061271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:24.079752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:24.170044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60167-4daf-4b10-99ba-9f9528d1ca96 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:24.203544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:24.204177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.204453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.204718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] PATH_INFO: `/v3/users/cfed0bd9414d41fb947727074728cd5a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.205894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] RBAC: Authorizing `identity:get_user(user_id=cfed0bd9414d41fb947727074728cd5a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:24.207320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:24.209048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 536/1077] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:24 2022] GET /identity/v3/users/cfed0bd9414d41fb947727074728cd5a => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:24.215542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:24.323492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbff553-dc14-4ed1-acdc-7e0fd8591b47 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:24.349407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:24.359226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:24.359875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.360157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.360429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.361550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:24.363002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fc47a-e0ec-47e2-b32c-93ae7a01d904 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:24.369664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 542/1078] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:24 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:24.371182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:24.416730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7516140-2fce-4b98-9f40-da59e1b534ba None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.417044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7516140-2fce-4b98-9f40-da59e1b534ba None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.417287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7516140-2fce-4b98-9f40-da59e1b534ba None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.418781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 543/1079] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:24.423601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.423889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.424163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.442865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:24.444063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:24.463123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:24.483217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276149-4fea-44c3-b173-cf22ddb5c09a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:24.516352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 544/1080] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:24.518304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:24.518969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.519248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.519524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae/users/cfed0bd9414d41fb947727074728cd5a/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:24.521723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] RBAC: Authorizing `identity:create_grant(project_id=22756dc870b34229a4747c67aa49a9ae, user_id=cfed0bd9414d41fb947727074728cd5a, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:24.523071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:24.537412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 537/1081] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:43:24 2022] PUT /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae/users/cfed0bd9414d41fb947727074728cd5a/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:43:24.786579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:24.892132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf0bf-4dcf-4460-9e8a-5707a2bbc50c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:24.918750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:24.929051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:24.929705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:24.929991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:24.930266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:25.001345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:25.002746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:25.110607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 545/1082] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:25.112840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:25.211932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcc37-5faa-4ba7-9718-ba88e081c31d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:25.245366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:25.246033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:25.246311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:25.246614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:25.257185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:25.258615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:25.260780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b7cd8dbf-7129-4ccd-90f2-8239b950495f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:25.274724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:25.276460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:25.277549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:25.277549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:25.277549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:25.279742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 538/1083] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:25.285496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:25.312322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:25.386875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ccd18-9e11-4a76-9dfe-828b830a4311 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:25.421412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:25.422052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:25.422328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:25.422745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:25.423420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:25.423728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:25.425074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d4dfd-14eb-4361-8595-8d0050d4f737 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:25.436466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 546/1084] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:25.892508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa47a0-5bd5-419c-8539-0cc0b50e38e8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:25.892788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa47a0-5bd5-419c-8539-0cc0b50e38e8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:25.893060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa47a0-5bd5-419c-8539-0cc0b50e38e8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:25.894453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 539/1085] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:25.899507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:25.899989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:25.900084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:25.920376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:25.921585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:25.987441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 547/1086] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:26.253274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa47a0-5bd5-419c-8539-0cc0b50e38e8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.356043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa47a0-5bd5-419c-8539-0cc0b50e38e8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.390568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:26.391204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.391481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.391755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.401938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-30b6112b-7059-4863-a3ed-b10257da1c89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:26.406845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:26.407781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:26.409051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 540/1087] 10.222.0.82 () {56 vars in 1120 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:26.414246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.431495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:26.466616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb9b3-d4ba-41cf-8643-56578af89be8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.466910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb9b3-d4ba-41cf-8643-56578af89be8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.467191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb9b3-d4ba-41cf-8643-56578af89be8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.468522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 541/1088] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:26.473321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.473614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.473908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.491541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:26.492734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:26.514702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed97b4-77d6-4c5d-b4fd-ca208469cd75 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.542948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:26.549106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:26.549743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.550019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.550288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.550942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:26.551258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:26.552600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:26.560658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 542/1089] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 21:43:26.565423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 548/1090] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:26.566505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.663553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422cdc-af57-4f4f-9c34-60575a4d9074 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.697751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:26.698382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.698660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.698931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.708915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:26.710382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:26.724895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:26.725938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:26.726820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:26.726820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:26.726820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:26.726820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 543/1091] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:26.732367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.809059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:26.829242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.830527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae803a-55d4-4e37-b0a5-545d0a032ef5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.865010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:26.865201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:26.865787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.866064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.866335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.867230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:26.867534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:26.869076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:26.881314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 549/1092] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:26.886964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:26.934465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04a58-6bc7-471f-995a-ada78660fd5a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.970108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:26.970742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:26.971018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:26.971288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:26.982799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:26.988034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:26.989156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:26.990060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:26.990723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1d3ec-0c23-4b4c-9b2a-5f69f769c709 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:26.991126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 544/1093] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:26.995138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.024946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.026202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.026487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.026755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.033823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.035248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.037258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:43:27.047620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:27.048621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:27.050135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 550/1094] 10.222.0.82 () {56 vars in 1093 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:27.054367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.090481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.096590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fb0bd-938d-43fc-85df-fbde36f581e6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.131986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.132677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.132960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.133234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.133925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:27.134232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.136219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.145017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 545/1095] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.150733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.156867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d936f81-5a14-485f-b233-2c307959d010 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.191578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.192195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.192477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.192748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.193469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:27.193807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.195225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.213606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 551/1096] 10.222.0.82 () {56 vars in 1133 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.219269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.250576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a704e0e-9f27-4497-b2a2-f7ab31694e3b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.277114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.286291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.286927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.287187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.287461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] PATH_INFO: `/v3/users/cfed0bd9414d41fb947727074728cd5a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.288603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cfed0bd9414d41fb947727074728cd5a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.290069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.291683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 546/1097] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/users/cfed0bd9414d41fb947727074728cd5a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.297378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.324500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.327716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc036d5-1478-4333-ab5d-38c5f4434a34 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.361127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.361691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.361983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.362253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.395565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c139-a55a-4db5-96b9-f453b26a95d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.428957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.429572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.429878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.430150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.430487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.432549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.485337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.499715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.501162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.529751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.532327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 552/1098] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:27.537921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.605560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 547/1099] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:27.611889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ffcae-c341-4bf7-9d64-c91d899e0d23 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.672726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.673360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.673662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.673941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.683657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.685090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:27.700119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:27.701284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:43:27.702588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:27.702588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:43:27.702588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:27.705812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 553/1100] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:27.706937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.714334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd21d7-1de6-44e3-8d81-7c8234518189 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.718963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.739873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.749201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.749867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.750144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.750419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.751546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.752989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.754662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 548/1101] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.761528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.811465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed6e4-e9a7-4c11-b83a-abfeabaff027 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.844713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.845331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.845630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.845915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.846555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:27.846861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.848220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.859872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675bd41-9d46-4c1d-a75e-693cb00be162 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:27.860024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 554/1102] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.865809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:27.894078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:27.894680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:27.894958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:27.895231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:27.895948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=22756dc870b34229a4747c67aa49a9ae, user.id=cfed0bd9414d41fb947727074728cd5a) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:27.896256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:27.897729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:27.903644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 549/1103] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/role_assignments?user.id=cfed0bd9414d41fb947727074728cd5a&scope.project.id=22756dc870b34229a4747c67aa49a9ae&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:43:27.937397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:27.973735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4e770-d947-452d-a630-e03b20bd14ae None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:28.010879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:28.011612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:28.011791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:28.012057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:28.019445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:28.020899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:43:28.032887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:28.034326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:43:28.035260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:28.037853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 555/1104] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:43:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:28.040111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:28.101022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:28.143798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c97cc-2900-4ad4-b1e5-5a148f183085 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:28.176892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:28.177493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:28.177793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:28.178061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:28.178741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:28.179040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:28.180459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307c43-38c6-40bb-995a-99dddf27bfed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:28.192832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 550/1105] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:28.198770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:28.297318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8677c7-19bf-4465-aa22-ebf81d5a4a07 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:28.322822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:28.331130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:28.331741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:28.332022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:28.332296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:28.333012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] RBAC: Adding query filter params (role.id=04d1aad1801248d5a677e6420baee70b, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=d441753da1af4f3c8f3b5fb4088d201b) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:28.333322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:28.334772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14e5b0-190f-4281-b4d8-6a60df3fe5be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:28.340822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 556/1106] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:28 2022] GET /identity/v3/role_assignments?user.id=d441753da1af4f3c8f3b5fb4088d201b&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=04d1aad1801248d5a677e6420baee70b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:29.821608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135947e7-460b-43ac-bfc4-202d712b24e5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:29.822239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135947e7-460b-43ac-bfc4-202d712b24e5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:29.822239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135947e7-460b-43ac-bfc4-202d712b24e5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:29.823519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 551/1107] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:29.828804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:29.829337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:29.829337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:29.847100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:29.848284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:29.922578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 557/1108] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:30.188122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135947e7-460b-43ac-bfc4-202d712b24e5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:30.286935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135947e7-460b-43ac-bfc4-202d712b24e5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:30.303873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:30.320326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:30.320940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:30.321212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:30.321480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:30.322634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:30.324056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980fd44-1728-41ba-a899-67684af12e6d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:30.325589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 552/1109] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:30.331410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:30.428792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233faa9-211f-4777-a32c-79f2777ff99f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:30.451010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:30.463382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:30.463994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:30.464276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:30.464551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:30.465207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:30.466672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e9529-f530-4d42-9ef6-0a2909d3e135 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:30.487314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 558/1110] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:43:30 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:32.392384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551041a-f777-4350-b82c-7a00aedf3c67 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:32.392917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551041a-f777-4350-b82c-7a00aedf3c67 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:32.393006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551041a-f777-4350-b82c-7a00aedf3c67 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:32.394418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 553/1111] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:32.399635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:32.400027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:32.400204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:32.418115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:32.419310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:32.483749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 559/1112] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:32.750879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551041a-f777-4350-b82c-7a00aedf3c67 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:32.849469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551041a-f777-4350-b82c-7a00aedf3c67 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:32.883435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:32.884054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:32.884335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:32.884638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] PATH_INFO: `/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:32.886321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] RBAC: Authorizing `identity:get_role(role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:32.887440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:32.888743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 554/1113] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:32 2022] GET /identity/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:32.894653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:32.990000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2102d-af1f-4925-bf3f-0fe1a0587157 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:32.999528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:33.024347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:33.024973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:33.025282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:33.025556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] PATH_INFO: `/v3/users/ed7b7b1818d74585bb791de96473f030` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:33.038602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] RBAC: Authorizing `identity:get_user(user_id=ed7b7b1818d74585bb791de96473f030)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:33.040033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:33.042183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 560/1114] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:32 2022] GET /identity/v3/users/ed7b7b1818d74585bb791de96473f030 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:33.048172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:33.102015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:33.145161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69ba7-672b-476a-8311-e797a9a77e87 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:33.179198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:33.179791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:33.180064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:33.180333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:33.246149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:33.247816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:33.302760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:33.347594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 555/1115] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:33.353529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:33.449570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d486f1-8546-4616-b76d-0bc3f107bd6b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:33.481947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:33.482558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:33.482831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:33.483100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:33.484177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:33.485745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2824b84-6ff1-4aed-8ba2-1ff3caf1cf41 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:33.487335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 561/1116] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:33 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:33.493671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:33.593798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531790-1acc-43fe-9158-1263e1cb312d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:33.626152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:33.626752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:33.627029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:33.627296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:33.628005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=22756dc870b34229a4747c67aa49a9ae, user.id=ed7b7b1818d74585bb791de96473f030) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:33.628312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:33.629753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bf74-bedc-44d4-af6d-b13e3a1dc59b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:33.635398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 556/1117] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:33 2022] GET /identity/v3/role_assignments?user.id=ed7b7b1818d74585bb791de96473f030&scope.project.id=22756dc870b34229a4747c67aa49a9ae&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:35.512049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695153d5-3b7a-4a98-ab1a-46121858c819 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:35.512577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695153d5-3b7a-4a98-ab1a-46121858c819 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:35.512577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695153d5-3b7a-4a98-ab1a-46121858c819 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:35.513964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 562/1118] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:35.519284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:35.519685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:35.519837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:35.537733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:35.538921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:35.551677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:35.605941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 557/1119] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:35.871205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695153d5-3b7a-4a98-ab1a-46121858c819 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:35.908673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:35.974104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695153d5-3b7a-4a98-ab1a-46121858c819 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:36.007339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:36.007963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:36.008249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:36.008524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] PATH_INFO: `/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:36.009962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] RBAC: Authorizing `identity:get_role(role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:36.011320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:36.012624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 563/1120] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:35 2022] GET /identity/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:36.018590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:36.114784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41cc2-8712-4115-b31e-01426ddbfd2d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:36.147335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:36.147938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:36.148176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:36.148448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] PATH_INFO: `/v3/users/ed7b7b1818d74585bb791de96473f030` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:36.149564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] RBAC: Authorizing `identity:get_user(user_id=ed7b7b1818d74585bb791de96473f030)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:36.151554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:36.153135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 558/1121] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:36 2022] GET /identity/v3/users/ed7b7b1818d74585bb791de96473f030 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:36.159169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:36.257971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6725-7f69-4e57-8745-5b5ecb53b1da None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:36.259573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:36.293784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:36.294409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:36.294683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:36.294951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:36.296045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:36.297462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0de5f-908d-4488-b7ef-46b330c9f827 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:36.299123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 564/1122] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:36 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:36.305134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:36.323321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:36.402400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911679-13a0-4c24-8fa9-17a41f3414ed None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:36.434888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:36.435532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:36.435810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:36.436081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae/users/ed7b7b1818d74585bb791de96473f030/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:36.438668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=22756dc870b34229a4747c67aa49a9ae, user_id=ed7b7b1818d74585bb791de96473f030, role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:36.439667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d6066-5bbd-4efe-8bcf-1f65189bd7bf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:36.453045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 559/1123] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:43:36 2022] PUT /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae/users/ed7b7b1818d74585bb791de96473f030/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:43:38.311360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28975-0f02-43d6-b5f4-4b4ccb13e84c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:38.311917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28975-0f02-43d6-b5f4-4b4ccb13e84c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:38.311917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28975-0f02-43d6-b5f4-4b4ccb13e84c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:38.313246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 565/1124] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:38.318492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:38.318878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:38.319051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:38.337269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:38.338480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:38.402784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 560/1125] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:38.676144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b28975-0f02-43d6-b5f4-4b4ccb13e84c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:38.775466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b28975-0f02-43d6-b5f4-4b4ccb13e84c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:38.808931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:38.809554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:38.809864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:38.810138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] PATH_INFO: `/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:38.811524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] RBAC: Authorizing `identity:get_role(role_id=c2cbc36004574fa2a1e6b2f9aebb14ca)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:38.812886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:38.814213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 566/1126] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:38 2022] GET /identity/v3/roles/c2cbc36004574fa2a1e6b2f9aebb14ca => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:38.820150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:38.828210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:38.915647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a200c-af0a-4893-a14b-9acf6afe6fce None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:38.949429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:38.950013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:38.950288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:38.950554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] PATH_INFO: `/v3/users/ed7b7b1818d74585bb791de96473f030` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:38.951656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] RBAC: Authorizing `identity:get_user(user_id=ed7b7b1818d74585bb791de96473f030)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:38.953073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:38.954638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 561/1127] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:38 2022] GET /identity/v3/users/ed7b7b1818d74585bb791de96473f030 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:38.960358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:38.999940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:39.058946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b623b3-8e77-4c72-8e00-012cba85bed5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:39.091917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:39.092513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:39.092792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:39.093063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:39.157979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:39.159415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:39.195768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:39.255182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 567/1128] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:39.261191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:39.338512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:39.363719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c5b5f-cbfb-45f8-a26b-b00febc80755 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:39.395983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239854d-94d2-4056-9487-037642765a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:39.396606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239854d-94d2-4056-9487-037642765a92 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:39.396885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239854d-94d2-4056-9487-037642765a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:39.397154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239854d-94d2-4056-9487-037642765a92 None admin] PATH_INFO: `/v3/projects/22756dc870b34229a4747c67aa49a9ae` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:39.398229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239854d-94d2-4056-9487-037642765a92 None admin] RBAC: Authorizing `identity:get_project(project_id=22756dc870b34229a4747c67aa49a9ae)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:39.399655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239854d-94d2-4056-9487-037642765a92 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:39.401800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 562/1129] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:39 2022] GET /identity/v3/projects/22756dc870b34229a4747c67aa49a9ae => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:39.408288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:39.505253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc63c0-1049-4ba2-b279-af2af4000613 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:39.538898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:39.539496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:39.539777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:39.540051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:39.540798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] RBAC: Adding query filter params (role.id=c2cbc36004574fa2a1e6b2f9aebb14ca, scope.project.id=22756dc870b34229a4747c67aa49a9ae, user.id=ed7b7b1818d74585bb791de96473f030) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:39.541109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:39.542565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a531d-f655-4c8b-b9c7-fb562f615299 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:39.548212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 568/1130] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:39 2022] GET /identity/v3/role_assignments?user.id=ed7b7b1818d74585bb791de96473f030&scope.project.id=22756dc870b34229a4747c67aa49a9ae&role.id=c2cbc36004574fa2a1e6b2f9aebb14ca => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:41.435236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a59d6-e762-49e3-8867-e5b536daadc1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:41.435763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a59d6-e762-49e3-8867-e5b536daadc1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:41.435763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a59d6-e762-49e3-8867-e5b536daadc1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:41.437128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 563/1131] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:43:41.442386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:41.442807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:41.442947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:41.449846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:41.462314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:41.463545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:41.526796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 569/1132] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:41.793793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a59d6-e762-49e3-8867-e5b536daadc1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:41.895492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a59d6-e762-49e3-8867-e5b536daadc1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:41.928152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:41.928769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:41.929042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:41.929306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:41.930419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:41.932508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e089c-e827-4dd1-90a5-084a2aeb4236 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:41.934079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 564/1133] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:41.939945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:42.035828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea760f54-975f-4dd7-abda-8c099992ec7d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:42.064529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:42.069694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:42.070303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:42.070586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:42.070858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:42.071494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:42.072925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7740cf6-b9c5-454d-a7bc-95f37c24e4f7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:42.098891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 570/1134] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:43:41 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:44.024100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c169-47a0-4689-9b8f-1c205ecb9b41 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:44.024633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c169-47a0-4689-9b8f-1c205ecb9b41 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:44.024633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c169-47a0-4689-9b8f-1c205ecb9b41 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:44.026054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 565/1135] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:44.031635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:44.032029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:44.032187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:44.050301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:44.051481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:44.116432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 571/1136] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:44.383539 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7c169-47a0-4689-9b8f-1c205ecb9b41 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:44.395644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:44.484208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7c169-47a0-4689-9b8f-1c205ecb9b41 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:44.518210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:44.518832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:44.519111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:44.519384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:44.520475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:44.521942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e4ca-09ec-402b-b85a-58e9951b0d8b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:44.523504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 566/1137] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:43:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:44.529830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:44.625025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1018c08-e856-44b5-9681-f2fc3c404d88 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:44.639000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:44.659083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:44.659683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:44.659960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:44.660230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:44.660863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:44.662310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec6f7-a92d-49c2-ad00-58805770f0ee None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:44.683752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 572/1138] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:43:44 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:43:46.610624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73095dd-6969-4704-ab6f-2d3ac4280b15 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:46.611175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73095dd-6969-4704-ab6f-2d3ac4280b15 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:46.611175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73095dd-6969-4704-ab6f-2d3ac4280b15 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:46.612502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 567/1139] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:46.617635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:46.618060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:46.618184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:46.636331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:46.637505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:46.702537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 573/1140] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:46.970649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73095dd-6969-4704-ab6f-2d3ac4280b15 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.047931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:47.070199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73095dd-6969-4704-ab6f-2d3ac4280b15 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:47.104328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:47.104951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:47.105233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:47.105505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:47.115995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:47.121449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:47.122676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:47.123659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 568/1141] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:43:47.128843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.221590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:47.224487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaaac5-fcd4-4d8f-81e3-f817c68024f4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:47.260082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:47.260686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:47.260963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:47.261233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:47.261936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:47.262240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:47.263597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:47.271361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 574/1142] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:47.277259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.376476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bd742-5b52-4fa9-97d3-f3411dcc6e1c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:47.410173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:47.410796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:47.411073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:47.411344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] PATH_INFO: `/v3/users/7efaabe38d704a91ba5b9ad4b19539e0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:47.419483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:47.425929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] RBAC: Authorizing `identity:get_user(user_id=7efaabe38d704a91ba5b9ad4b19539e0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:47.427398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:47.429064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 569/1143] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:47 2022] GET /identity/v3/users/7efaabe38d704a91ba5b9ad4b19539e0 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:47.435521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.537021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e3396-160c-42b1-9ec7-ab364c3f5643 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:47.570036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:47.570635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:47.570915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:47.571189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:47.600724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:47.639410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:47.640861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:47.739730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 575/1144] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:47.746261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.842090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0522f47-b683-474c-899e-0213a68e7f97 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:47.875351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:47.875969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:47.876247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:47.876517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] PATH_INFO: `/v3/projects/206379d2eb434f77a61ab57625648815` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:47.877649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] RBAC: Authorizing `identity:get_project(project_id=206379d2eb434f77a61ab57625648815)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:47.879087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd02a96-aa58-49e6-9731-aa0fb1d3b309 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:47.880689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 570/1145] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:47 2022] GET /identity/v3/projects/206379d2eb434f77a61ab57625648815 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:47.886820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:47.946504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:47.984272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322bbea-c0b3-409c-8740-44f3198492f5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:48.017529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:48.018158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:48.018439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:48.018712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:48.019431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=206379d2eb434f77a61ab57625648815, user.id=7efaabe38d704a91ba5b9ad4b19539e0) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:48.019744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:48.021152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee95c6-f37c-473f-ba5f-69ecae5b8fb2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:48.026387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 576/1146] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:47 2022] GET /identity/v3/role_assignments?user.id=7efaabe38d704a91ba5b9ad4b19539e0&scope.project.id=206379d2eb434f77a61ab57625648815&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:49.907323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cf2c-9075-4cec-aecf-44d0d0ff4d11 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:49.907839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cf2c-9075-4cec-aecf-44d0d0ff4d11 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:49.907959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cf2c-9075-4cec-aecf-44d0d0ff4d11 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:49.909339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 571/1147] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:49.914507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:49.914929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:49.915055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:49.933300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:49.934524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:49.998470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 577/1148] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:50.265649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49cf2c-9075-4cec-aecf-44d0d0ff4d11 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:50.329025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:50.370241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49cf2c-9075-4cec-aecf-44d0d0ff4d11 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:50.404409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:50.405027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:50.405305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:50.405574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:50.415298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:50.420080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:50.421050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:50.422186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 572/1149] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:50.427414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:50.481609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:50.524908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f943d-b064-4f92-ad20-04d05aafb325 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:50.558376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:50.558983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:50.559265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:50.559539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:50.560200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:50.560507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:50.561897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:50.570072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 578/1150] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:50.576550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:50.673485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e68d05-fdd7-4ed4-ac13-6233afd18259 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:50.696946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:50.708170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:50.708792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:50.709065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:50.709331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] PATH_INFO: `/v3/users/7efaabe38d704a91ba5b9ad4b19539e0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:50.710465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] RBAC: Authorizing `identity:get_user(user_id=7efaabe38d704a91ba5b9ad4b19539e0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:50.711893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:50.713495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 573/1151] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:50 2022] GET /identity/v3/users/7efaabe38d704a91ba5b9ad4b19539e0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:50.719356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:50.815957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1571-3b15-4ebc-8c30-81e5972bb706 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:50.831778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:50.848354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:50.848967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:50.849245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:50.849515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] PATH_INFO: `/v3/projects/206379d2eb434f77a61ab57625648815` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:50.850629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] RBAC: Authorizing `identity:get_project(project_id=206379d2eb434f77a61ab57625648815)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:50.852049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149395c2-b0bf-44e8-9d36-5e598765fe8f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:50.854319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 579/1152] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:50 2022] GET /identity/v3/projects/206379d2eb434f77a61ab57625648815 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:50.860400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:50.955820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19808-ca77-484c-9494-874e8d9daca4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:50.988022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:50.988663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:50.988939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:50.989208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] PATH_INFO: `/v3/projects/206379d2eb434f77a61ab57625648815/users/7efaabe38d704a91ba5b9ad4b19539e0/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:50.991117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] RBAC: Authorizing `identity:create_grant(project_id=206379d2eb434f77a61ab57625648815, user_id=7efaabe38d704a91ba5b9ad4b19539e0, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:50.992541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7318b5-a49b-4593-a627-316a9993b995 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:51.006210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 574/1153] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:43:50 2022] PUT /identity/v3/projects/206379d2eb434f77a61ab57625648815/users/7efaabe38d704a91ba5b9ad4b19539e0/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:43:52.868730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d750-b664-453a-9801-02d2d9177e5d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:52.869271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d750-b664-453a-9801-02d2d9177e5d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:52.869271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d750-b664-453a-9801-02d2d9177e5d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:52.870648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 580/1154] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:52.875890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:52.876304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:52.876440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:52.894156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:52.895348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:52.943045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:52.964205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 575/1155] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:53.232706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d750-b664-453a-9801-02d2d9177e5d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:53.334440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d750-b664-453a-9801-02d2d9177e5d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:53.367485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:53.368103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:53.368382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:53.368676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:53.378871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:43:53.383464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:53.384547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:43:53.385705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 581/1156] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:53.390656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:53.486854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad92604-9038-4007-b31a-1134cbcfe399 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:53.518393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:53.518983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:53.519256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:53.519525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:53.520175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:53.520475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:53.521856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:53.530310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 576/1157] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:53.536116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:53.581667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:53.633775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9517693-1111-476d-9e06-8f799548bfbe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:53.666891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:53.667515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:53.667799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:53.668077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] PATH_INFO: `/v3/users/7efaabe38d704a91ba5b9ad4b19539e0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:53.669195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] RBAC: Authorizing `identity:get_user(user_id=7efaabe38d704a91ba5b9ad4b19539e0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:53.670651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:53.672212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 582/1158] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:43:53 2022] GET /identity/v3/users/7efaabe38d704a91ba5b9ad4b19539e0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:53.678071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:53.724347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:53.777380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9235d90-14e0-4a4b-8fe0-79d482a4e88c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:53.809491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:53.810118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:53.810399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:53.810672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:53.875139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:53.876581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:53.919799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:53.973558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 577/1159] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:53.979595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:54.077467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7cb63-0217-467d-a9d8-81506a77ad18 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:54.090448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:54.110419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:54.111049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:54.111332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:54.111613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] PATH_INFO: `/v3/projects/206379d2eb434f77a61ab57625648815` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:54.112733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] RBAC: Authorizing `identity:get_project(project_id=206379d2eb434f77a61ab57625648815)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:54.114196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5d4a7-8a44-42b5-9604-5ff72592bc75 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:54.115842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 583/1160] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:43:53 2022] GET /identity/v3/projects/206379d2eb434f77a61ab57625648815 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:54.122441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:54.217885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4d42-769e-4e9a-92dc-83ab83a1a47c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:54.250826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:54.251419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:54.251697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:54.251964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:54.252685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=206379d2eb434f77a61ab57625648815, user.id=7efaabe38d704a91ba5b9ad4b19539e0) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:54.252991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:54.254435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ba72-4f03-47dd-b1ec-ab51ba87ca2b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:54.260646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 578/1161] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:43:54 2022] GET /identity/v3/role_assignments?user.id=7efaabe38d704a91ba5b9ad4b19539e0&scope.project.id=206379d2eb434f77a61ab57625648815&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:56.140785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716278-bf02-42b1-8cdb-b4bde8e647df None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:56.141335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716278-bf02-42b1-8cdb-b4bde8e647df None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:56.141335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716278-bf02-42b1-8cdb-b4bde8e647df None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:56.142712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 584/1162] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:56.147875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:56.148283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:56.148417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:56.166658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:56.167821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:56.169759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:56.232563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 579/1163] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:56.497303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716278-bf02-42b1-8cdb-b4bde8e647df None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:56.601622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716278-bf02-42b1-8cdb-b4bde8e647df None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:56.635366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:56.635966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:56.636241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:56.636507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:56.684478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:56.702559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:56.704000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:56.800107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 585/1164] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:43:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:43:56.806138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:56.902136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227acb5-3a88-41d7-961c-67f445f27e17 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:56.935210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:56.935824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:56.936098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:56.936364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:56.942683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:56.944118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:56.945785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f9b2474f-51f7-462c-b80e-92b512f3ead5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:43:56.955889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:43:56.957124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:43:56.958222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 580/1165] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:43:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:43:56.963048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:57.042078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:43:57.060270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258bf0-c6de-4884-a252-6d1a15e96b0a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:57.092989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:57.093581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:57.093886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:57.094158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:57.094796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:43:57.095093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:57.096443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58eff-3622-45d4-8627-4e48a5c3c970 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:57.103938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 586/1166] 10.222.0.82 () {56 vars in 1115 bytes} [Thu Aug 18 21:43:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:43:58.971003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96112e7-70eb-40c4-88a0-03b3e4a15444 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:58.971501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96112e7-70eb-40c4-88a0-03b3e4a15444 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:58.971501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96112e7-70eb-40c4-88a0-03b3e4a15444 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:58.972803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 581/1167] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:43:58.977941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f741-4dfb-4bf6-9b4c-ab4b2d3b9588 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:58.978366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f741-4dfb-4bf6-9b4c-ab4b2d3b9588 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:58.978490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f741-4dfb-4bf6-9b4c-ab4b2d3b9588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:58.996108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9ba8f741-4dfb-4bf6-9b4c-ab4b2d3b9588 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:43:58.997272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9ba8f741-4dfb-4bf6-9b4c-ab4b2d3b9588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:43:59.061427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 587/1168] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:43:59.326701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96112e7-70eb-40c4-88a0-03b3e4a15444 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:43:59.426858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96112e7-70eb-40c4-88a0-03b3e4a15444 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:43:59.460322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:43:59.460920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:43:59.461199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:43:59.461469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:43:59.462160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:43:59.463524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a133c-4a3c-46b3-8195-dff92233192a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:43:59.479820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 582/1169] 10.222.0.82 () {60 vars in 1138 bytes} [Thu Aug 18 21:43:59 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:44:01.381833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb22472-87d9-4b36-b45c-cc8300e9c9d2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:01.382358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb22472-87d9-4b36-b45c-cc8300e9c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:01.382358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb22472-87d9-4b36-b45c-cc8300e9c9d2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:01.383712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 588/1170] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:44:01.389491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:01.389914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:01.390071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:01.409767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:44:01.410960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:44:01.462237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:01.476882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 583/1171] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:44:01.744595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb22472-87d9-4b36-b45c-cc8300e9c9d2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:01.820132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:01.843686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb22472-87d9-4b36-b45c-cc8300e9c9d2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:01.876460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:01.877085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:01.877363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:01.877661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] PATH_INFO: `/v3/domains/c53273de8a414014adabc703908e8b71` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:01.889872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c53273de8a414014adabc703908e8b71)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:01.891309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216292d6-bb0d-46c5-909e-d94ffb8fb256 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:01.892913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 589/1172] 10.222.0.82 () {56 vars in 1183 bytes} [Thu Aug 18 21:44:01 2022] GET /identity/v3/domains/c53273de8a414014adabc703908e8b71 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:01.898815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:01.995508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a098c0-3719-4f83-896b-4c0f6ae387d3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:02.027366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:02.027970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:02.028248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:02.028546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:02.029427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:02.030896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eabc0-a0bd-4a39-90ce-d87fe119d8db None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:02.057105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 584/1173] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:44:01 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:44:03.955935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab6f5-21dd-4aa7-ab88-f433ec5bda1a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:03.956453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab6f5-21dd-4aa7-ab88-f433ec5bda1a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:03.956571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab6f5-21dd-4aa7-ab88-f433ec5bda1a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:03.957841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 590/1174] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:44:03.963033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:03.963425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:03.963581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:03.981290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:44:03.982449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:44:04.016362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:04.049210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 585/1175] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:44:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:44:04.320278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab6f5-21dd-4aa7-ab88-f433ec5bda1a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:04.419758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab6f5-21dd-4aa7-ab88-f433ec5bda1a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:04.452831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:04.453460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:04.453899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:04.454180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] PATH_INFO: `/v3/domains/c53273de8a414014adabc703908e8b71` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:04.455302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=c53273de8a414014adabc703908e8b71)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:04.456735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb93f-636a-468f-a210-3f08ae2b4ead None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:04.458322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 591/1176] 10.222.0.82 () {56 vars in 1183 bytes} [Thu Aug 18 21:44:04 2022] GET /identity/v3/domains/c53273de8a414014adabc703908e8b71 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:04.464149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:04.562750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ef63b-88d1-4c15-a807-93b58c19378b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:04.594984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:04.595523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:04.595794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:04.596058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:04.596699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:04.598151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9a79c-4723-47d3-98b3-7e79f283f0f7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:04.620752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 586/1177] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:44:04 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:44:06.573800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67fbb1-b4a2-4de5-b28d-de05c33137e8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:06.574331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67fbb1-b4a2-4de5-b28d-de05c33137e8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:06.574331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67fbb1-b4a2-4de5-b28d-de05c33137e8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:06.575667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 592/1178] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:44:06.580748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:06.581207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:06.581300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:06.599324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:44:06.600511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:44:06.624163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:06.672485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 587/1179] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:44:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:44:06.946319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67fbb1-b4a2-4de5-b28d-de05c33137e8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:06.950149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.045769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67fbb1-b4a2-4de5-b28d-de05c33137e8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.078159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.078783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.079064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.079341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.089687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:44:07.094434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:44:07.095640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:44:07.096834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 593/1180] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:44:07.101750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:07.198292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70509bf6-d420-44a2-8513-5278aebe88e6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.231169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.231774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.232054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.232328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.232990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:44:07.233297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:07.234716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:07.242568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 588/1181] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:44:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:07.248624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:07.312409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.349799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0691b9-29ff-49a0-b89e-1b6d9593b62f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.381886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.382501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.382779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.383047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] PATH_INFO: `/v3/users/414b5e9392cc48bcae29fb7f0668c222` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.395944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] RBAC: Authorizing `identity:get_user(user_id=414b5e9392cc48bcae29fb7f0668c222)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:07.397378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:07.398992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 594/1182] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:44:07 2022] GET /identity/v3/users/414b5e9392cc48bcae29fb7f0668c222 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:07.405813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:07.424558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.501284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda488-789f-4081-9619-505eef87e313 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.533876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.534468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.534744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.535015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.599633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:07.601064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:07.617823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.698411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 589/1183] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:44:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:44:07.704371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:07.800446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9db19a-11d0-4215-8a4d-0e16cc1f8a60 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.823101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.834605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.835227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.835503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.835773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] PATH_INFO: `/v3/projects/ab2c835c8717400c927da124c8051d43` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.836871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] RBAC: Authorizing `identity:get_project(project_id=ab2c835c8717400c927da124c8051d43)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:07.838416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9081a79-10d0-49b8-80e6-97cc63f7d036 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:07.840030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 595/1184] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:44:07 2022] GET /identity/v3/projects/ab2c835c8717400c927da124c8051d43 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:07.846340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:07.942538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdae22-800a-4a91-84f6-acdcb931f20b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:07.959760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:07.976239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:07.976838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:07.977117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:07.977388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:07.978145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=ab2c835c8717400c927da124c8051d43, user.id=414b5e9392cc48bcae29fb7f0668c222) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:44:07.978451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:07.980390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6525-6c9a-465d-b8a1-98a70f63370c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:07.985650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 590/1185] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:44:07 2022] GET /identity/v3/role_assignments?user.id=414b5e9392cc48bcae29fb7f0668c222&scope.project.id=ab2c835c8717400c927da124c8051d43&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:09.841838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f035-815e-487e-b6f7-2a98fa4d9f70 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:09.842383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f035-815e-487e-b6f7-2a98fa4d9f70 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:09.842383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f035-815e-487e-b6f7-2a98fa4d9f70 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:09.843726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 596/1186] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:44:09.849002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:09.849396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:09.849544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:09.867098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:44:09.868282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:44:09.933393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 591/1187] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:44:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:44:10.201750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914f035-815e-487e-b6f7-2a98fa4d9f70 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:10.301109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914f035-815e-487e-b6f7-2a98fa4d9f70 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:10.333228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:10.333864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:10.334141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:10.334410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:10.344620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:44:10.349171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:44:10.350284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:44:10.351217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 597/1188] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:44:10.356833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:10.452972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbaaea-3e3a-4383-b63d-53da4e3e645d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:10.465631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:10.486271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:10.486874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:10.487154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:10.487425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:10.488087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:44:10.488389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:10.489780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:10.498136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 592/1189] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:10.503963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:10.553962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:10.603194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97440640-6e36-4a76-a88e-ea7fb146eda4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:10.636058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:10.636672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:10.636955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:10.637229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] PATH_INFO: `/v3/users/414b5e9392cc48bcae29fb7f0668c222` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:10.638374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] RBAC: Authorizing `identity:get_user(user_id=414b5e9392cc48bcae29fb7f0668c222)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:10.639798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:10.641355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 598/1190] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:44:10 2022] GET /identity/v3/users/414b5e9392cc48bcae29fb7f0668c222 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:10.647332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:10.746104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97077a5-77fc-4734-8e14-33d547aa4b65 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:10.778215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:10.778838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:10.779116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:10.779652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] PATH_INFO: `/v3/projects/ab2c835c8717400c927da124c8051d43` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:10.780756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] RBAC: Authorizing `identity:get_project(project_id=ab2c835c8717400c927da124c8051d43)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:10.782227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf2107-ccdd-48c0-be9b-6fad8af94680 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:10.783831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 593/1191] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:44:10 2022] GET /identity/v3/projects/ab2c835c8717400c927da124c8051d43 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:10.789999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:10.886560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8bfe-b724-4bc9-a288-b06e99ca9def None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:10.903999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:10.921391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:10.922071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:10.922353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:10.922628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] PATH_INFO: `/v3/projects/ab2c835c8717400c927da124c8051d43/users/414b5e9392cc48bcae29fb7f0668c222/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:10.924505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab2c835c8717400c927da124c8051d43, user_id=414b5e9392cc48bcae29fb7f0668c222, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:10.926068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c31b7-73a1-4b4c-a39a-73d820b8e738 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:10.937963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 599/1192] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:44:10 2022] PUT /identity/v3/projects/ab2c835c8717400c927da124c8051d43/users/414b5e9392cc48bcae29fb7f0668c222/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:44:12.810639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67133c66-9a0e-4399-ad6f-056317fe45a2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:12.811184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67133c66-9a0e-4399-ad6f-056317fe45a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:12.811184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67133c66-9a0e-4399-ad6f-056317fe45a2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:12.812555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 594/1193] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:44:12.817672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:12.818109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:12.818213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:12.836411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:44:12.837582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:44:12.901791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 600/1194] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:44:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:44:13.165025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67133c66-9a0e-4399-ad6f-056317fe45a2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:13.208827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:13.271074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67133c66-9a0e-4399-ad6f-056317fe45a2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:13.304299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:13.304917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:13.305195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:13.305466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:13.315278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:44:13.319918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:44:13.321028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:44:13.322761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 595/1195] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:44:13.327675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:13.422907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534034fd-fa1c-4232-ba84-c21bfa35fbb1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:13.431721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:13.457198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:13.457835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:13.458118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:13.458392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:13.459055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:44:13.459362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:13.460718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:13.469231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 601/1196] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:13.474858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:13.574781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382202c-521a-4a11-8536-ed3744c6b64c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:13.576296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:13.609725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:13.610240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:13.610520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:13.610800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] PATH_INFO: `/v3/users/414b5e9392cc48bcae29fb7f0668c222` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:13.611941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=414b5e9392cc48bcae29fb7f0668c222)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:13.613376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:13.614971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 596/1197] 10.222.0.82 () {56 vars in 1177 bytes} [Thu Aug 18 21:44:13 2022] GET /identity/v3/users/414b5e9392cc48bcae29fb7f0668c222 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:13.621451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:13.718666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2bc1a-9a50-45bc-bdb2-2079a6dc986a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:13.751151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:13.751783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:13.752062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:13.752332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:13.789930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:13.819760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:13.821199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:13.923184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 602/1198] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:44:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:44:13.929089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:14.026441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701518-f7c3-4a16-aeda-47658e168ce3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:14.059892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:14.060509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:14.060785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:14.061059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] PATH_INFO: `/v3/projects/ab2c835c8717400c927da124c8051d43` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:14.062178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] RBAC: Authorizing `identity:get_project(project_id=ab2c835c8717400c927da124c8051d43)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:14.063627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174ec1a-68d0-4a7c-83b2-f4e9ee62b9ca None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:14.065237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 597/1199] 10.222.0.82 () {56 vars in 1186 bytes} [Thu Aug 18 21:44:13 2022] GET /identity/v3/projects/ab2c835c8717400c927da124c8051d43 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:14.071353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:44:14.138113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:14.168471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04236aff-2674-48e9-881f-59d20373e543 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:44:14.202692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:44:14.203299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:44:14.203580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:44:14.203854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:44:14.204575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=ab2c835c8717400c927da124c8051d43, user.id=414b5e9392cc48bcae29fb7f0668c222) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:44:14.204882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:44:14.206330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b3bb1-4861-427d-aa33-97743a66f397 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:44:14.211957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 603/1200] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:44:14 2022] GET /identity/v3/role_assignments?user.id=414b5e9392cc48bcae29fb7f0668c222&scope.project.id=ab2c835c8717400c927da124c8051d43&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:44:24.055200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:44:24.209378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=33 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:09.408059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870192f-562f-4b20-9e39-0ff7a013deb3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:09.408723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870192f-562f-4b20-9e39-0ff7a013deb3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:09.408723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870192f-562f-4b20-9e39-0ff7a013deb3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:09.410227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 598/1201] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:45:09.417375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:09.417974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:09.418064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:09.440410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:09.441983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:09.530786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 604/1202] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:09.831876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870192f-562f-4b20-9e39-0ff7a013deb3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:09.960646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870192f-562f-4b20-9e39-0ff7a013deb3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:10.001676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:10.002234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:10.003175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:10.003175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:10.020673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:10.020673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:10.041335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:10.043633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:10.044819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:10.044819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:10.044819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:10.044819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 599/1203] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:10.049643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:10.167875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1601f1-f9c8-4985-8ba2-f88abbbd6bfe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:10.204630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:10.211873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:10.213166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:10.213481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:10.213778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:10.214456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:10.214766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:10.216472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64422bb3-1829-4a87-8576-ec29cdc10e2c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:10.230953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 605/1204] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:10.237641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:10.298097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:10.356108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f757210-d1a6-450d-bbd0-8f0c0d56157e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:10.397911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:10.398575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:10.398872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:10.399146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:10.400326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:10.401898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eab13-aec8-4ad0-a589-992c4997d0e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:10.428860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 600/1205] 10.222.0.82 () {60 vars in 1132 bytes} [Thu Aug 18 21:45:10 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:45:12.443566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024789b5-18ee-4734-b549-10fb8738efb8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:12.444109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024789b5-18ee-4734-b549-10fb8738efb8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:12.444109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024789b5-18ee-4734-b549-10fb8738efb8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:12.445628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 606/1206] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:12.451695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:12.452193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:12.452293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:12.470755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:12.471949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:12.565446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 601/1207] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:12.839255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024789b5-18ee-4734-b549-10fb8738efb8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:12.973484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024789b5-18ee-4734-b549-10fb8738efb8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:13.022654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:13.023289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:13.023571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:13.023846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:13.035828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:13.040658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:13.041725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:13.042572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 607/1208] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:45:13.049770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:13.093368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:13.169685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0e44b-b231-4bae-9705-f7b9e3e32b37 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:13.211251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:13.211918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:13.212233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:13.212514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:13.213188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:13.213497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:13.214916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:13.224517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 602/1209] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:13.230335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:13.289127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:13.353393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527566fa-761a-4e96-a1a7-de5d077fdc98 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:13.393346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:13.394375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:13.394852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:13.395357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:13.408729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:13.411424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:13.435192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:13.436414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:13.437473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:13.437473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:13.437473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:13.437473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 608/1210] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:13.442930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:13.567372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:13.572514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcbaf0-9bd0-4e41-b341-b673cae493f3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:13.616200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:13.616772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:13.617055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:13.617805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:13.618181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:13.619087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:13.620477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:13.638482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 603/1211] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:45:13.640069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:13.770528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ce655-9897-4e4e-8c71-033620f931e7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:13.773437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:13.808382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:13.809010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:13.809295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:13.809568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:13.818900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:13.820276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:13.833157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:13.834469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:13.835446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 609/1212] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:13.841762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:13.974797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a10e21-1874-440d-92db-a98de0924e25 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:14.011927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:14.012506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:14.012789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:14.013063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:14.013829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:14.014214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:14.015844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:14.035271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 604/1213] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:14.040984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:14.159772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976975a-82d5-42a7-820b-8bcf5892c069 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:14.203215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:14.203927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:14.204204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:14.204476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:14.237111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:14.287343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:14.288907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:14.420271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 610/1214] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:45:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 21:45:14.426068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:14.449251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:14.548253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30962261-e44c-4c04-bd90-b61b24acd5d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:14.590507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:14.591368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:14.591368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:14.591635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:14.604770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:14.606708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:14.627290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:14.628677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:14.629838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:14.629838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:14.629838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:14.629838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 605/1215] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:14.635304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:14.713133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:14.760300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58da7f-7467-4969-8a50-2f1858f7b5f3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:14.802851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:14.803583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:14.803761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:14.804006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:14.804754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:14.805063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:14.806516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:14.820821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 611/1216] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:14.833844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:14.910609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:14.947243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36f89-d590-4e80-b1ff-6c7e4779b8fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:14.986549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:14.987288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:14.987544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:14.987868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:14.997412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:14.997412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:15.010350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:15.011529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:15.012611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 606/1217] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:15.017673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:15.132602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d870cd-7a50-4446-abf6-764f9cf1fa96 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:15.158390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:15.171661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:15.172299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:15.172608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:15.172911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:15.173696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:15.174059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:15.175913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e45c5-827c-40ff-9f32-d19c9ca6cb3d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:15.189063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 612/1218] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:15.195398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:15.330504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9466a6d-a25c-467c-8468-9490586bb648 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:15.360056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:15.366353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:15.367070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:15.367457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:15.367847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:15.368858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=6ebbb44da1cc45218ce9bb43f11e6715) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:15.369278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:15.370885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46046f-370f-4fb9-8c41-cdf2801e66f3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:15.376657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 607/1219] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:45:15 2022] GET /identity/v3/role_assignments?user.id=6ebbb44da1cc45218ce9bb43f11e6715&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:17.363018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e68ea-440a-45af-a40c-a768f4400c17 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:17.363571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e68ea-440a-45af-a40c-a768f4400c17 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:17.363571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e68ea-440a-45af-a40c-a768f4400c17 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:17.364963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 613/1220] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:17.370151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:17.370566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:17.370699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:17.389932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:17.391279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:17.464230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 608/1221] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:17.763094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e68ea-440a-45af-a40c-a768f4400c17 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:17.882034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e68ea-440a-45af-a40c-a768f4400c17 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:17.919864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:17.920509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:17.920823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:17.921075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:17.936058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:17.940838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:17.942385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:17.943663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 614/1222] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:17.948140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.069863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8557dd-3693-4e87-8c3a-a6d0cba72855 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:18.101806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:18.116469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:18.117225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:18.117534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:18.117999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:18.118756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:18.119259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:18.120799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:18.130534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 609/1223] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:18.137087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.186424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:18.253035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7390-e8e7-4bab-a908-516cb8d61622 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:18.294660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:18.295282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:18.295565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:18.295837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:18.307689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:18.309154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:18.324577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:18.325687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:18.326573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:18.326573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:18.326573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:18.326573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 615/1224] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:18.331970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.440120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0282e54-924b-4f9d-bbd7-fa17531509ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:18.475898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:18.476506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:18.476787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:18.477060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:18.477727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:18.478031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:18.479433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:18.492791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 610/1225] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:18.499282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.600456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:18.607853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341343-33e3-4e8f-8e1d-00d7d9d76d22 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:18.646151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:18.646788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:18.647101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:18.647376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:18.654871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:18.656551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:18.668751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:18.670086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:18.671086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 616/1226] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:18.676326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.731488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:18.805128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f3b03-ddac-4339-9251-656651a62f9a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:18.851988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:18.853129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:18.853444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:18.853773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:18.854711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:18.855022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:18.856852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:18.877509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 611/1227] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:18.883352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:18.991787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d923f3d-1230-44c4-8956-d9d88c4e8977 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:19.031008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:19.031653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:19.031937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:19.032208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:19.044585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:19.046089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:19.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:19.061711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:19.062600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:19.062600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:19.062600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:19.062600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 617/1228] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:19.067927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:19.176119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce233556-8c57-4544-8f72-394faaa9db28 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:19.179215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:19.217037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:19.217868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:19.218266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:19.218625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:19.219453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:19.219836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:19.221328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:19.235041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 612/1229] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:19.240813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:19.251487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:19.362004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14f08-4fd9-4278-995d-35dfebf73c99 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:19.398789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:19.400043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:19.400422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:19.400701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:19.409571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:19.411056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:19.428304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:19.429407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:19.430456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 618/1230] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:19.435359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:19.550430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ce9b-4587-42fe-a7cd-6d103dbd03e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:19.592611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:19.593231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:19.593505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:19.593858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:19.594582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:19.594901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:19.596519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0befb8-ae7c-4c52-aff8-c07b1d18700b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:19.609795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 613/1231] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:45:19.615722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:19.687212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:19.726102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225cda2-3b71-4706-9dfe-155f37a97bf0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:19.762006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:19.762751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:19.763087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:19.763400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/6ebbb44da1cc45218ce9bb43f11e6715/roles/0983328bf53b474a9ae3ff90884f68c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:19.780784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=6ebbb44da1cc45218ce9bb43f11e6715, role_id=0983328bf53b474a9ae3ff90884f68c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:19.782402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d3acd-8f20-4fed-bef7-c98bde36fe5d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:19.794846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 619/1232] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:45:19 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/6ebbb44da1cc45218ce9bb43f11e6715/roles/0983328bf53b474a9ae3ff90884f68c3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:45:21.706454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fd88-7c71-4e2f-9c9b-670bb7c5f088 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:21.706977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fd88-7c71-4e2f-9c9b-670bb7c5f088 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:21.706977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fd88-7c71-4e2f-9c9b-670bb7c5f088 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:21.708481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 614/1233] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:21.713519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:21.713974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:21.714104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:21.732340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:21.733536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:21.822653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 620/1234] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:22.103907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fd88-7c71-4e2f-9c9b-670bb7c5f088 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:22.132654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:22.217621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fd88-7c71-4e2f-9c9b-670bb7c5f088 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:22.255389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:22.256032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:22.256313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:22.256583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:22.268519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:22.273779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:22.275434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:22.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 615/1235] 10.222.0.82 () {56 vars in 1102 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:22.281897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:22.330219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:22.386364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa03373-7843-4148-8fb3-73d84bd49d43 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:22.422280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:22.422892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:22.423242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:22.423519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:22.424294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:22.424700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:22.426298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:22.435564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 621/1236] 10.222.0.82 () {56 vars in 1103 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:22.442094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:22.533730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:22.554364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426e15-34af-4a62-a610-ffa13158d700 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:22.589340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:22.589944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:22.590223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:22.590491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:22.600776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:22.602442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:22.619741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:22.620882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:22.626744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:22.626744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:22.626744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:22.626744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 616/1237] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:22.627007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:22.736143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c1fa-bcb7-4867-a767-45a63e6a60ad None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:22.745856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:22.772842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:22.773463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:22.773811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:22.774061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:22.774813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:22.775137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:22.776579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:22.788969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 622/1238] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:22.795378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:22.911476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5dcb-3d0d-4672-9f67-2d1886e309a8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:22.949347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:22.950020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:22.950295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:22.950567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:22.953714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:22.960237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:22.962590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:22.976572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:22.978581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:22.979759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 617/1239] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:22.984588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:23.095181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1b39-7d39-4f71-ac18-20d4f10846dc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:23.131802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:23.132414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:23.132692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:23.132960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:23.133726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:23.134057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:23.135563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:23.153483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 623/1240] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:23.159774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:23.270338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eace4-6f05-4811-805e-a301efe67eb0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:23.316421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:23.317269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:23.317699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:23.318039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:23.392178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:23.400539 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:23.402327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:23.516137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 618/1241] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:45:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:45:23.523079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:23.555408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:23.640689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e59cf7-60f3-4474-926b-9ba8555001b3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:23.676567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:23.677223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:23.677501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:23.677797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:23.688379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:23.689861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:23.705407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:23.706619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:23.707682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:23.707682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:23.707682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:23.707682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 624/1242] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:23.712596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:23.815876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:23.823231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506ac2-8fd2-457e-99aa-dd28d40dc38d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:23.859821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:23.860531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:23.860817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:23.861128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:23.861843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:23.862155 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:23.863528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:23.877650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 619/1243] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:23.881971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:23.971591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:23.997019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000c54e-57b1-4839-90ea-8f5dff704883 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:24.032392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:24.033022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:24.033300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:24.033572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:24.040661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:24.042139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:24.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:24.055396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:24.056553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 625/1244] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:24.062592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:24.177696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a83c3-abf6-41b1-9e96-f471fd915c28 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:24.215644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:24.216409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:24.216788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:24.217135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:24.218000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:24.218308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:24.219904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd522ea-39b1-436a-a34b-801459a7e4e7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:24.234152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 620/1245] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:24.240345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:24.361311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a59cb-9a39-4ce0-a618-9071549ee186 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:24.398877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:24.399499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:24.399781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:24.400066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:24.400788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] RBAC: Adding query filter params (role.id=0983328bf53b474a9ae3ff90884f68c3, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=6ebbb44da1cc45218ce9bb43f11e6715) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:24.401100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:24.402560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a756a3-9f8d-4388-9fda-168542d53577 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:24.409480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 626/1246] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:45:24 2022] GET /identity/v3/role_assignments?user.id=6ebbb44da1cc45218ce9bb43f11e6715&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=0983328bf53b474a9ae3ff90884f68c3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:26.475479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fa611-a792-47cc-a843-5c6ccf17627e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:26.476035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fa611-a792-47cc-a843-5c6ccf17627e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:26.476035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fa611-a792-47cc-a843-5c6ccf17627e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:26.477661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 621/1247] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:26.483588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:26.484082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:26.484196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:26.490701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:26.503542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:26.504775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:26.581085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 627/1248] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:26.874051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fa611-a792-47cc-a843-5c6ccf17627e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:26.893226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:26.988615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fa611-a792-47cc-a843-5c6ccf17627e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.025755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.027167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.027167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.027167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.040724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:27.046137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:27.047109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:27.048194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 622/1249] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:27.053969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:27.169917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0eb76-a947-4261-b538-4aca70c9a8b3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.203491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:27.209295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.209946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.210229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.210505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.211183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:27.211523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:27.213715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:27.225172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 628/1250] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:45:27.232065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:27.326073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:27.353481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31632-863b-43ff-a74f-d2f81fd93386 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.390855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.392457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.392985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.393348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.407103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:27.408862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:27.425216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:27.426447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:27.427425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:27.427425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:27.427425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:27.427425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 623/1251] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:45:27.433064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:27.553202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae56a-5b24-4fbe-ad8c-6b18913b9cf8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.592791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.593460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.593791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.594103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.594812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:27.595117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:27.596657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:27.610020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 629/1252] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:27.616107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:27.740844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf29df0-a1fa-4705-9020-798b05053468 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.771979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:27.782918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.783586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.783995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.784209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.792860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:27.794485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:27.808247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:27.809441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:27.810606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:27.811181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 624/1253] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:27.817272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:27.878748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:27.928683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec568-66b0-4d8a-b1ed-4adc2f4af9e7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:27.965575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:27.966260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:27.966508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:27.966798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:27.967679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:27.967953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:27.969505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:27.993781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 630/1254] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:27.998166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:28.120304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf62d54-08af-46e9-a2f7-8270046c511a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:28.159222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:28.159834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:28.160111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:28.160462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:28.226173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:28.241563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:28.243081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:28.369113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 625/1255] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:45:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:45:28.375520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:28.502298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fbb70-a1ee-498a-a823-178097b78339 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:28.515988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:28.543305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:28.544328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:28.544614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:28.544957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:28.557079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:28.558563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:28.573459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:28.574626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:28.575898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:28.575898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:28.575898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:28.575898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 631/1256] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 21:45:28.580403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:28.679759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:28.693520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc8dc-7fe2-42f8-936f-35ee7dab05f2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:28.732079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:28.732728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:28.732975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:28.733250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:28.733925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:28.734232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:28.735605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:28.747081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 626/1257] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:28.752730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:28.862735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e29291-66d0-4c58-ac46-30620dd46d6c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:28.901148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:28.901881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:28.902195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:28.902516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:28.910481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:28.912097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:28.924250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:28.925257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:28.931551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 632/1258] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:28.932884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:29.037058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a23087-e05e-4807-b838-cd837a10f333 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:29.074387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:29.074998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:29.075317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:29.075649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:29.076354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:29.076656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:29.078254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9de2c-0690-4046-9d9d-f34894257c32 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:29.091103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 627/1259] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:29.097247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:29.123429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:29.204929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48504662-5707-4a01-90ee-745e2032c851 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:29.239338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:29.239955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:29.240232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:29.240504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:29.241227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=6ebbb44da1cc45218ce9bb43f11e6715) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:29.241534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:29.242988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d4bb4-38fb-4583-a800-15ca507f707d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:29.248944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 633/1260] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:45:29 2022] GET /identity/v3/role_assignments?user.id=6ebbb44da1cc45218ce9bb43f11e6715&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:31.153139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1f56f-bf41-4aa9-8120-8b3d00174de1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:31.153710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1f56f-bf41-4aa9-8120-8b3d00174de1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:31.153710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1f56f-bf41-4aa9-8120-8b3d00174de1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:31.155186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 628/1261] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:31.160460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:31.160856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:31.161008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:31.178758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:31.179947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:31.245426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 634/1262] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:31.525067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1f56f-bf41-4aa9-8120-8b3d00174de1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:31.536214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:31.624910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1f56f-bf41-4aa9-8120-8b3d00174de1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:31.659533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:31.660150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:31.660425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:31.660691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:31.670443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:31.675197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:31.676310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:31.677370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 629/1263] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:31.682311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:31.700650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:31.781083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6bf99-32da-4157-a6e3-e4668bc3c39d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:31.813333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:31.813966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:31.814246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:31.814527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:31.815187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:31.815493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:31.816850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:31.825389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 635/1264] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:31.831122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:31.896847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:31.929518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c609b7-e830-470b-ad6e-6c3ab88c6443 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:31.971467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:31.972099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:31.972378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:31.972651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:31.982301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:31.983775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:31.998291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:31.999369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:32.000663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:32.000663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:32.000663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:32.000663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 630/1265] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:32.005255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.084227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:32.102489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d86f-ebc8-49df-ae33-65ab6d07fe55 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.134890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.135481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.135756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.136025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.136651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:32.136950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.138329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.149946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 636/1266] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:32.156047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.253542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21802bf5-2582-4e8d-9df6-d36f9ab12993 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.273967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:32.290747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.291460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.291728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.292000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.299813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.301348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:32.313498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:32.314663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:32.315609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 631/1267] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:32.321250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.418443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a838-b0f9-4ebb-aa1f-c830dfb7b83f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.452009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.452605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.452886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.453160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.453912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:32.454217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.455633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.463992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:32.473558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 637/1268] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:45:32.479815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.577135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b0c5d-d022-400d-b02e-16233a605477 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.609431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.610076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.610357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.610627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.621327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.622797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:32.636633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:32.637727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:32.638905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:32.638905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:32.638905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:32.638905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 632/1269] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:32.643518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.741906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d282d-27db-4348-918c-6e09fb21d49c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.774677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.775294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.775574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.775845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.776482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:32.776785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.778165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.789506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 638/1270] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:32.795801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:32.826373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:32.893520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c38f6b-a538-4d3b-bca3-875ccfa58165 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:32.927325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:32.927945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:32.928232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:32.928501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:32.934855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:32.936454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:32.947312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:32.948361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:32.949637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 633/1271] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:32.954416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:33.010888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:33.052475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d27-5979-440d-9121-49586df2b240 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:33.086220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:33.087431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:33.087708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:33.087978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:33.088664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:33.088968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:33.090404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f76ec-50e9-41aa-a2bc-8e8cbf974f23 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:33.102041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 639/1272] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:33.108085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:33.197857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:33.207101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f58ec-def5-4096-9561-a714cb639a94 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:33.239928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:33.240571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:33.240847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:33.241119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] PATH_INFO: `/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/6ebbb44da1cc45218ce9bb43f11e6715/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:33.243023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] RBAC: Authorizing `identity:create_grant(project_id=e30a6241ad1344af9a89bf9f50eb994e, user_id=6ebbb44da1cc45218ce9bb43f11e6715, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:33.244461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fcbf2-a67d-42cc-b4a3-9ebd7356d26b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:33.256849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 634/1273] 10.222.0.82 () {58 vars in 1439 bytes} [Thu Aug 18 21:45:33 2022] PUT /identity/v3/projects/e30a6241ad1344af9a89bf9f50eb994e/users/6ebbb44da1cc45218ce9bb43f11e6715/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:45:35.125213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7106e9-d68f-41da-8346-f25c27509043 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:35.125839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7106e9-d68f-41da-8346-f25c27509043 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:35.125839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7106e9-d68f-41da-8346-f25c27509043 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:35.127179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 640/1274] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:35.132399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:35.132839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:35.132934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:35.151646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:35.152846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:35.219818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 635/1275] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:35.495768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7106e9-d68f-41da-8346-f25c27509043 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:35.599076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7106e9-d68f-41da-8346-f25c27509043 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:35.616051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:35.632975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:35.633614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:35.633904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:35.634173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:35.643951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 21:45:35.648565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:35.649733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:35.650588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 641/1276] 10.222.0.82 () {56 vars in 1096 bytes} [Thu Aug 18 21:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:35.656293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:35.735911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:35.754354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c29bf-cd77-406e-848b-8b500383bc26 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:35.786957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:35.787558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:35.787835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:35.788105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:35.788761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:35.789065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:35.791362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:35.799470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 636/1277] 10.222.0.82 () {56 vars in 1099 bytes} [Thu Aug 18 21:45:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:35.805381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:35.904024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14228-db9f-4a84-8d86-541754074a24 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:35.937609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:35.938234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:35.938516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:35.938790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:35.948494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:35.949962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:35.965580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:35.966695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:35.967940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:35.967940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:35.967940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:35.967940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 642/1278] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:35.972620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:36.069601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09a257-a9a6-4e0f-9d4a-d644fb7d6df0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:36.102466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:36.103062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:36.103341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:36.103626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:36.104251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:36.104556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:36.105953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:36.117538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 637/1279] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:36.123092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:36.155993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:36.223959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8042d-bf3b-4bf7-83e4-8d4b6f3a4229 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:36.255996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:36.256619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:36.256901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:36.257176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:36.265141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:36.266598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 21:45:36.278339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:36.279391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:36.280998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 643/1280] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:36.286110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:36.290965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:36.391089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152847-0b10-4fe1-a08d-8dd610b4f423 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:36.423822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:36.424408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:36.424688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:36.424954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:36.425695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:36.425994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:36.427416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:36.444107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 638/1281] 10.222.0.82 () {56 vars in 1143 bytes} [Thu Aug 18 21:45:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:36.450615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:36.538553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:36.548198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ee1e-fb8b-448b-ae80-e657e275b74f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:36.581010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:36.581636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:36.582021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:36.582300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:36.646140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:36.647510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:36.732424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:36.742804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 644/1282] 10.222.0.82 () {58 vars in 1282 bytes} [Thu Aug 18 21:45:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 21:45:36.749071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:36.809208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:36.846430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ed80-6697-4039-97df-4216ae021a6a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:36.880532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:36.881149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:36.881426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:36.881727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:36.891417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:36.892872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 21:45:36.907216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:36.908339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 21:45:36.909253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 21:45:36.909253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 21:45:36.909253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:36.909253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 639/1283] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:36.914141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:37.009498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e1ab-9da4-4142-b0f1-3598bfa6667c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:37.042726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:37.043328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:37.043610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:37.043884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:37.044517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:37.044821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:37.046219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:37.057646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 645/1284] 10.222.0.82 () {56 vars in 1109 bytes} [Thu Aug 18 21:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:37.063474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:37.161381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388041b-46a6-42c6-aa9c-b0ee3ec9da08 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:37.175250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:37.196195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:37.196831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:37.197104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:37.197373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:37.203772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:37.205222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 21:45:37.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:37.217745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 18 21:45:37.218965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 640/1285] 10.222.0.82 () {56 vars in 1111 bytes} [Thu Aug 18 21:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:37.223766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:37.277035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:37.331574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6b94-072f-4366-93b7-5c2870e612b1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:37.364413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:37.365011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:37.365292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:37.365567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:37.366283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:37.366591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:37.368070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90a0e-1bc0-4334-98cf-1cf4d6508ab4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:37.379573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 646/1286] 10.222.0.82 () {56 vars in 1148 bytes} [Thu Aug 18 21:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:37.385870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:37.484506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a3cd-de58-4b51-8bd0-39a3b4c5afec None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:37.516003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:37.516588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:37.516861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:37.517127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:37.517864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] RBAC: Adding query filter params (role.id=a8f69336710f474db0973fac38e02b80, scope.project.id=e30a6241ad1344af9a89bf9f50eb994e, user.id=6ebbb44da1cc45218ce9bb43f11e6715) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:37.518173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:37.519588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25584c-0886-4f2c-ac3e-fbd9edab5589 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:37.525702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 641/1287] 10.222.0.82 () {56 vars in 1374 bytes} [Thu Aug 18 21:45:37 2022] GET /identity/v3/role_assignments?user.id=6ebbb44da1cc45218ce9bb43f11e6715&scope.project.id=e30a6241ad1344af9a89bf9f50eb994e&role.id=a8f69336710f474db0973fac38e02b80 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:39.434230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b857-8540-4e83-945e-072619050d9a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:39.434849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b857-8540-4e83-945e-072619050d9a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:39.434849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2b857-8540-4e83-945e-072619050d9a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:39.436082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 647/1288] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:39.441098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:39.441545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:39.441682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:39.459932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:39.461130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:39.489416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:39.526779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 642/1289] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:39.793258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2b857-8540-4e83-945e-072619050d9a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:39.891422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2b857-8540-4e83-945e-072619050d9a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:39.901803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:39.925141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:39.925791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:39.926072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:39.926347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:39.926925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:39.928277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:39.937229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:39.938397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:39.939737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 648/1290] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:45:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:39.944340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:40.039391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d788ac-43a2-41dd-8be9-fbfb5e73e31e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:40.072274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:40.072863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:40.073137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:40.073407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:40.074047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:40.074346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:40.075697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b19711-7297-4164-8eb5-3e6acb7a2e88 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:40.080281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 643/1291] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 21:45:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:40.085914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:40.183168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeba8b8-8bd2-4f43-bf6a-111c6bc32a9d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:40.215570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:40.216164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:40.216438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:40.216710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:40.217328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:40.217926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:40.219022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f802e9-1179-43c7-a860-1f7a47f2c66e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:40.224884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 649/1292] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 21:45:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:42.107087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea552c-b7e6-40f7-b49f-6b268d48082a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:42.107675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea552c-b7e6-40f7-b49f-6b268d48082a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:42.107675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea552c-b7e6-40f7-b49f-6b268d48082a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:42.108996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 644/1293] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:42.114228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a912b0-48d6-4e66-9ac7-19501c33f474 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:42.114623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a912b0-48d6-4e66-9ac7-19501c33f474 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:42.114781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a912b0-48d6-4e66-9ac7-19501c33f474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:42.132466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-25a912b0-48d6-4e66-9ac7-19501c33f474 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:42.134175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25a912b0-48d6-4e66-9ac7-19501c33f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:42.163279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:42.201715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 650/1294] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:42.473132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea552c-b7e6-40f7-b49f-6b268d48082a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:42.503948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:42.571438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea552c-b7e6-40f7-b49f-6b268d48082a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:42.604878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:42.605470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:42.605774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:42.606047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:42.606685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:42.608039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e029b2-05b4-4b2b-8c57-01ccafea33cd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:42.621880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 645/1295] 10.222.0.82 () {60 vars in 1141 bytes} [Thu Aug 18 21:45:42 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:45:44.518761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5704-a824-4bb1-921d-b28d7de979ed None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:44.519289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5704-a824-4bb1-921d-b28d7de979ed None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:44.519289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5704-a824-4bb1-921d-b28d7de979ed None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:44.520657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 651/1296] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:44.526306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:44.526752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:44.526853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:44.544700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:44.545906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:44.615770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 646/1297] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:44.885408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb5704-a824-4bb1-921d-b28d7de979ed None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:44.983919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb5704-a824-4bb1-921d-b28d7de979ed None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:45.017304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:45.017952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:45.018235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:45.018508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:45.019086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:45.020443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:45.028896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:45.030047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:45.031248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 652/1298] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:45:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:45.036348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:45.054750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:45.135924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec0faf-73d4-4dff-a3cf-3ec1b65c8265 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:45.168750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:45.169349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:45.169649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:45.169921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:45.170550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:45.170852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:45.172215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:45.177168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 647/1299] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 21:45:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:45.183930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:45.215167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:45.290448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7092e9f-d3b0-46db-9e64-58fe01635122 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:45.328958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:45.329449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:45.329761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:45.330032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:45.330728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] RBAC: Adding query filter params (interface=public, service_id=2a8c01fa1b3d4753b151fdb3be601fdf, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:45.331023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:45.332412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b590fc-c950-4771-a8c0-f72cca573e31 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:45.343578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 653/1300] 10.222.0.82 () {56 vars in 1251 bytes} [Thu Aug 18 21:45:45 2022] GET /identity/v3/endpoints?service_id=2a8c01fa1b3d4753b151fdb3be601fdf&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:45.344835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:45.423976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:45.441924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40dbda2-a74f-4006-8f2a-ada604825826 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:45.474320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:45.474912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:45.475184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:45.475449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:45.476035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:45.478017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333af0a-34da-4885-9001-177bb2792e8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:45.484594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 648/1301] 10.222.0.82 () {56 vars in 1087 bytes} [Thu Aug 18 21:45:45 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:45:47.380662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6e60-8d02-4ae4-81b5-0795236321cb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:47.381220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6e60-8d02-4ae4-81b5-0795236321cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:47.381220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6e60-8d02-4ae4-81b5-0795236321cb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:47.382589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 654/1302] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:45:47.387649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:47.388055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:47.388199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:47.405833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:45:47.407268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:45:47.475361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 649/1303] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:45:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 21:45:47.745384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd6e60-8d02-4ae4-81b5-0795236321cb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:47.837968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:47.845330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd6e60-8d02-4ae4-81b5-0795236321cb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:47.880866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:47.881491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:47.881797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:47.882571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:47.882698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:47.884569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 21:45:47.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 21:45:47.894894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Aug 18 21:45:47.896134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 655/1304] 10.222.0.82 () {56 vars in 1117 bytes} [Thu Aug 18 21:45:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 21:45:47.900890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:47.982318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:48.006353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e9214-0c5c-41e9-9175-4cd224229104 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:48.042008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:48.042625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:48.042904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:48.043177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:48.043810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:45:48.044131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:48.045509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d3999-f863-4c4e-8db1-c3a9aae46025 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:48.050812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 650/1305] 10.222.0.82 () {56 vars in 1116 bytes} [Thu Aug 18 21:45:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:45:48.057384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:45:48.155874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f16db-c596-4c5d-b956-c2716ce87ffe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:45:48.189587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:45:48.190221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:45:48.190502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:45:48.190773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:45:48.191427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:45:48.192892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5b4c3-b930-49e2-a33d-fb4eadad1a41 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:45:48.212090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 656/1306] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:45:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:45:58.047709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=30 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:45:58.208003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:52.680010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3853d0-4d90-4e22-8640-0eb3f0b625d9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:52.681111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3853d0-4d90-4e22-8640-0eb3f0b625d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:52.681636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3853d0-4d90-4e22-8640-0eb3f0b625d9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:52.687679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 651/1307] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:46:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:46:52.690717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb574-b43d-40aa-9a39-2fbb20add14a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:52.690717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb574-b43d-40aa-9a39-2fbb20add14a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:52.690717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb574-b43d-40aa-9a39-2fbb20add14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:52.855931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7e8cb574-b43d-40aa-9a39-2fbb20add14a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:46:52.857486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7e8cb574-b43d-40aa-9a39-2fbb20add14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:53.066455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 657/1308] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:46:53.362172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:53.362851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:53.363348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:53.365194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 652/1309] 10.222.0.82 () {54 vars in 965 bytes} [Thu Aug 18 21:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:46:53.372128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c40f7-aa78-4287-b7bc-05a466fadd08 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:53.372721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c40f7-aa78-4287-b7bc-05a466fadd08 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:53.373238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c40f7-aa78-4287-b7bc-05a466fadd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:53.457254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-346c40f7-aa78-4287-b7bc-05a466fadd08 None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:46:53.458622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-346c40f7-aa78-4287-b7bc-05a466fadd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:53.480290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:53.551828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 658/1310] 10.222.0.82 () {58 vars in 1062 bytes} [Thu Aug 18 21:46:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:46:53.561710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:46:53.564008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:53.698192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c19776-8fc8-4345-b0dd-707a4b0d74fd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:46:53.786649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:46:53.787263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:53.787544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:53.787815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:53.790076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:53.849192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:53.909840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:46:53.911295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c2dbb-25f9-4f3b-9b09-7c89c82921f1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:46:53.937006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 653/1311] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:46:56.183498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:56.184083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:56.184083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:56.185443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 659/1312] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:46:56.191181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872166ce-6c07-4843-ab09-8033fb87f535 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:56.191671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872166ce-6c07-4843-ab09-8033fb87f535 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:56.191868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872166ce-6c07-4843-ab09-8033fb87f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:56.266622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-872166ce-6c07-4843-ab09-8033fb87f535 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:46:56.267880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-872166ce-6c07-4843-ab09-8033fb87f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:56.365046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 654/1313] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:46:56.645709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:46:56.647242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:56.753727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf0c8f-37fe-4cf7-844a-9ce0dad401e1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:46:56.824675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:56.827332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:46:56.827752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:56.828225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:56.828671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:56.830516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:56.935179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:46:56.936603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c726e-7260-40b5-843e-0ebd4c6dfa3c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:46:56.955553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 660/1314] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:46:59.155243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:59.155777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:59.155777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:59.157402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 655/1315] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:46:59.162645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02be7-61a3-4219-9ff8-72ca3d66b747 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:59.163081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02be7-61a3-4219-9ff8-72ca3d66b747 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:59.163211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02be7-61a3-4219-9ff8-72ca3d66b747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:59.228874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f6a02be7-61a3-4219-9ff8-72ca3d66b747 None None] MFA Rules not processed for user `1058c3362abc419ba2002e2ebf5395c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:46:59.230090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f6a02be7-61a3-4219-9ff8-72ca3d66b747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:59.283919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:59.328134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 661/1316] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:46:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:46:59.613927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:46:59.615496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:59.625233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:59.723098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b90b94-aabc-4767-8799-daffbc1ea028 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:46:59.795013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:46:59.795608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:46:59.795885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:46:59.796157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:46:59.797792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:46:59.859416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:46:59.905735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:46:59.907428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5a0a1-1a95-41eb-931f-7576d0fbf5cf service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:46:59.925215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 656/1317] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:46:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:02.367384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e174-d0e1-4fca-a984-4ad1b635ffb9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:02.367945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e174-d0e1-4fca-a984-4ad1b635ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:02.367945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e174-d0e1-4fca-a984-4ad1b635ffb9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:02.369276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 662/1318] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:02.374450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105d8d-5a2f-44a9-8ec2-b220820e8d68 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:02.374866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105d8d-5a2f-44a9-8ec2-b220820e8d68 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:02.374982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105d8d-5a2f-44a9-8ec2-b220820e8d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:02.439534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-82105d8d-5a2f-44a9-8ec2-b220820e8d68 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:02.440734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-82105d8d-5a2f-44a9-8ec2-b220820e8d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:02.537891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 657/1319] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:02.846647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:02.846931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:02.847200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:02.848523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 663/1320] 10.222.0.82 () {54 vars in 965 bytes} [Thu Aug 18 21:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:02.854895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e856-027b-485a-8e33-efdc738fc579 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:02.855183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e856-027b-485a-8e33-efdc738fc579 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:02.855458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e856-027b-485a-8e33-efdc738fc579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:02.876680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:02.930510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0871e856-027b-485a-8e33-efdc738fc579 None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:02.931724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0871e856-027b-485a-8e33-efdc738fc579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:03.008945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 658/1321] 10.222.0.82 () {58 vars in 1062 bytes} [Thu Aug 18 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:03.018127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:03.019673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:03.128821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7511-8b92-4e1f-a94e-74f3c6553b40 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:03.203521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:03.204132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:03.204632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:03.204759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:03.206310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:03.213921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:03.314213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:03.315925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4709633-bad6-434f-b441-2eb5f88cf1e3 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:03.334293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 664/1322] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:06.369803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:06.370375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:06.370375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:06.371677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 659/1323] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:06.376958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0499e-bb1d-4b3c-b432-ddd7787c6013 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:06.377380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0499e-bb1d-4b3c-b432-ddd7787c6013 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:06.377526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0499e-bb1d-4b3c-b432-ddd7787c6013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:06.442439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1ea0499e-bb1d-4b3c-b432-ddd7787c6013 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:06.443634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1ea0499e-bb1d-4b3c-b432-ddd7787c6013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:06.504506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:06.544982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 665/1324] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:06.829964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:06.831535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:06.938877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c77da8-9617-464b-b862-9cc5be442a84 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:06.952117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:07.013920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:07.014423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:07.014699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:07.014967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:07.016607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:07.124175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:07.125681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b3687-6811-4675-aa1e-550fa3223386 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:07.144931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 660/1325] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:09.117822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:09.118349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:09.118349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:09.119637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 666/1326] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:09.124774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0adba-f1c6-46aa-89c6-9c16a90c579f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:09.125193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0adba-f1c6-46aa-89c6-9c16a90c579f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:09.125372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0adba-f1c6-46aa-89c6-9c16a90c579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:09.173743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:09.192867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cef0adba-f1c6-46aa-89c6-9c16a90c579f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:09.194103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cef0adba-f1c6-46aa-89c6-9c16a90c579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:09.292244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 661/1327] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:09.573462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:09.575065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:09.688380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16fd1c-2cdb-4bbf-b375-ee561ada235e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:09.763923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:09.764530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:09.764810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:09.765082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:09.766718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:09.784338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:09.879345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:09.880790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753028f-21b8-40b3-8354-60694010e246 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:09.899369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 667/1328] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:11.880771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:11.881401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:11.881401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:11.883240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 662/1329] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:11.888339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769115f7-cbb1-408d-a237-d7d83c04eee9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:11.888749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769115f7-cbb1-408d-a237-d7d83c04eee9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:11.888893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769115f7-cbb1-408d-a237-d7d83c04eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:11.954182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-769115f7-cbb1-408d-a237-d7d83c04eee9 None None] MFA Rules not processed for user `1058c3362abc419ba2002e2ebf5395c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:11.955382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-769115f7-cbb1-408d-a237-d7d83c04eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:12.018534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:12.051088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 668/1330] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:12.337285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:12.338907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:12.441282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f4a71-d84e-4c32-9aeb-bcc4bef4b70d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:12.461068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:12.520585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:12.521213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:12.521493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:12.521800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:12.523490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:12.638880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:12.640640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28615c-dbf3-4242-8ad4-f4fba9e17b79 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:12.660015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 663/1331] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:14.838586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:14.839142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:14.839142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:14.840478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 669/1332] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:14.846019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d93a-dc27-4c70-885b-fdca3a9427b1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:14.846433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d93a-dc27-4c70-885b-fdca3a9427b1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:14.846566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d93a-dc27-4c70-885b-fdca3a9427b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:14.909533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:14.912448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c5f5d93a-dc27-4c70-885b-fdca3a9427b1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:14.914797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c5f5d93a-dc27-4c70-885b-fdca3a9427b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:15.011891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 664/1333] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:15.295651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:15.297176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:15.399087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc1756-537b-449c-8343-e7ad552b7462 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:15.471174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:15.471780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:15.472065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:15.472338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:15.473972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:15.500983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:15.583428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:15.584782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cd77f-4c72-4cbd-9d97-92567511c36f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:15.603449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 670/1334] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:17.535483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:17.536012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:17.536012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:17.537388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 665/1335] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:17.543160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66d01-cabb-4434-ac8c-480a8348554c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:17.543605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66d01-cabb-4434-ac8c-480a8348554c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:17.543699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66d01-cabb-4434-ac8c-480a8348554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:17.609737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-09a66d01-cabb-4434-ac8c-480a8348554c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:17.610941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-09a66d01-cabb-4434-ac8c-480a8348554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:17.683961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:17.706804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 671/1336] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:47:18.021277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:18.022868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:18.124919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd6588-6159-41a5-8207-3a208a8f2a5c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:18.152609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:18.198844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:18.199439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:18.199716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:18.199987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:18.201590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:18.308091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:18.309544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3ea7f-326c-4600-9382-e9f09b175018 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:18.327506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 666/1337] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:20.551019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:20.551520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:20.551631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:20.552904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 672/1338] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:20.558497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e014c88-2c2f-4061-9c93-92f33d41cc5a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:20.558931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e014c88-2c2f-4061-9c93-92f33d41cc5a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:20.559037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e014c88-2c2f-4061-9c93-92f33d41cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:20.623771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8e014c88-2c2f-4061-9c93-92f33d41cc5a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:20.624979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8e014c88-2c2f-4061-9c93-92f33d41cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:20.633771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:20.722968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 667/1339] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:21.006434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:21.008565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:21.110721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345595a-0d2a-41ba-8c30-eefff05fa412 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:21.185620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:21.186268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:21.186550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:21.186825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:21.188450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:21.225078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:21.299623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:21.301052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8371aaf-3d52-42a1-b5aa-f3a0dfb723c0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:21.318709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 673/1340] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:25.239334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:25.239825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:25.239941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:25.241249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 668/1341] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:47:25.246602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a211-66ce-49dd-940b-53582ad6ea7d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:25.247027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a211-66ce-49dd-940b-53582ad6ea7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:25.247287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a211-66ce-49dd-940b-53582ad6ea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:25.316016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0eb8a211-66ce-49dd-940b-53582ad6ea7d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:25.317233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0eb8a211-66ce-49dd-940b-53582ad6ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:25.405728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:25.418200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 674/1342] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:25.698532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:25.700056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:25.809970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac8c89-b0b9-4c04-b897-be2e173530df None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:25.849110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:25.886284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:25.886916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:25.887224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:25.887502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:25.889104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:26.000179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:26.001542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf2bb1-918a-4f57-97fc-22587d7c93e5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:26.020667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 669/1343] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:28.534257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:28.534791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:28.534791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:28.536160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 675/1344] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:28.542428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10ad8b-32fa-44b4-b90c-21d8d74445f1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:28.542985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10ad8b-32fa-44b4-b90c-21d8d74445f1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:28.542985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10ad8b-32fa-44b4-b90c-21d8d74445f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:28.613879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cc10ad8b-32fa-44b4-b90c-21d8d74445f1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:28.615111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cc10ad8b-32fa-44b4-b90c-21d8d74445f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:28.633079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:28.719647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 670/1345] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:29.002865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:29.004403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:29.111635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69f385-a4c6-4e6e-b062-8f7e1224c97f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:29.187745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:29.188358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:29.188636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:29.188907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:29.190542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:29.239436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:29.317363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:29.319041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa96711-f650-4370-8b11-97f5486bd07d service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:29.338438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 676/1346] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:33.269255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:33.269916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:33.269916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:33.271494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 671/1347] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:33.277636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c5b3-587b-4d2a-89f7-52751a82b601 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:33.278104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c5b3-587b-4d2a-89f7-52751a82b601 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:33.278202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3c5b3-587b-4d2a-89f7-52751a82b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:33.360273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0cb3c5b3-587b-4d2a-89f7-52751a82b601 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:33.361493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0cb3c5b3-587b-4d2a-89f7-52751a82b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:33.463470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:33.466550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 677/1348] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:33.752378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:33.753988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:33.866183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc035c3-a22d-4a5a-823e-6330c423bb7f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:33.914686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:33.941354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:33.942049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:33.942328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:33.942595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:33.944199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:34.058460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:34.059893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627c343-4014-4096-b2c9-19d5ef03ae93 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:34.078510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 672/1349] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:38.921927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f5210-8e30-476a-b0e1-be5b69c7d9fe None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:38.922410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f5210-8e30-476a-b0e1-be5b69c7d9fe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:38.922520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f5210-8e30-476a-b0e1-be5b69c7d9fe None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:38.923814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 678/1350] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:38.932734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e206-e244-47e1-a151-0625ee6248e5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:38.933241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e206-e244-47e1-a151-0625ee6248e5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:38.933395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e206-e244-47e1-a151-0625ee6248e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:39.030115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-79e3e206-e244-47e1-a151-0625ee6248e5 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:39.031571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79e3e206-e244-47e1-a151-0625ee6248e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:39.052547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:39.179973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 673/1351] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:47:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:43.393563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b651-04d3-446f-a4b3-d8964570c790 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:43.394159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b651-04d3-446f-a4b3-d8964570c790 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:43.394159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b651-04d3-446f-a4b3-d8964570c790 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:43.395555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 679/1352] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:43.400980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e465-b27b-45d1-a271-0d973701abe3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:43.401445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e465-b27b-45d1-a271-0d973701abe3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:43.401771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e465-b27b-45d1-a271-0d973701abe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:43.483989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aa94e465-b27b-45d1-a271-0d973701abe3 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:43.485275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa94e465-b27b-45d1-a271-0d973701abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:43.562878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:43.602519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 674/1353] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:45.096220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661013c-be1d-474f-bd0f-fd960f2549c5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:45.096747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661013c-be1d-474f-bd0f-fd960f2549c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:45.097051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661013c-be1d-474f-bd0f-fd960f2549c5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:45.098497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 680/1354] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:45.104521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e19e-a625-4fa9-9553-0b25e90f62bd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:45.104967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e19e-a625-4fa9-9553-0b25e90f62bd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:45.105204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e19e-a625-4fa9-9553-0b25e90f62bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:45.182458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9103e19e-a625-4fa9-9553-0b25e90f62bd None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:45.183649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9103e19e-a625-4fa9-9553-0b25e90f62bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:45.294071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 675/1355] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:47:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:48.571920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1fc6f-102f-493f-805d-af9eda31c431 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:48.572454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1fc6f-102f-493f-805d-af9eda31c431 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:48.572551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1fc6f-102f-493f-805d-af9eda31c431 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:48.573848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 681/1356] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:48.579147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2b3c7-0ebd-4062-8d39-5530f0cc16a5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:48.579568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2b3c7-0ebd-4062-8d39-5530f0cc16a5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:48.579708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2b3c7-0ebd-4062-8d39-5530f0cc16a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:48.623602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:48.667254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aaa2b3c7-0ebd-4062-8d39-5530f0cc16a5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:48.668444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aaa2b3c7-0ebd-4062-8d39-5530f0cc16a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:48.807425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 676/1357] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:47:49.107778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:49.108060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:49.108735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:49.110103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 682/1358] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 21:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:49.118998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aadc6a-50e6-49ec-94a4-ed660f8705f0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:49.119286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aadc6a-50e6-49ec-94a4-ed660f8705f0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:49.119564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aadc6a-50e6-49ec-94a4-ed660f8705f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:49.219686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-01aadc6a-50e6-49ec-94a4-ed660f8705f0 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:49.220899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-01aadc6a-50e6-49ec-94a4-ed660f8705f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:49.283152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:49.356315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 677/1359] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 21:47:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:49.363947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:49.365467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:49.518229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbf3c2-e8d1-4b99-8d2e-39eb2dfb6010 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:49.598701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:49.642787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:49.643590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:49.643951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:49.644298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:49.646151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:49.741451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:49.743170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a347c1-c863-48b1-94f3-1dcea7bb1d7d service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:49.761571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 683/1360] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:51.893738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:51.894400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:51.894400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:51.898365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 678/1361] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:51.901231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eecea9-714f-443c-b6f5-c8b718d7617b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:51.901976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eecea9-714f-443c-b6f5-c8b718d7617b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:51.901976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eecea9-714f-443c-b6f5-c8b718d7617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:51.990861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-56eecea9-714f-443c-b6f5-c8b718d7617b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:51.992536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-56eecea9-714f-443c-b6f5-c8b718d7617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:52.048598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:52.132519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 684/1362] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:52.425898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:52.427406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:52.593840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a6076-d74f-40b6-a933-57624e3d870f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:52.604215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:52.726662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:52.727368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:52.727718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:52.728058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:52.729770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:52.829060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:52.830711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1a1e-e74e-42d2-9f32-d07d0678d0d3 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:52.850251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 679/1363] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:55.099496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4dca-0c14-4a82-98a1-9aca7a3e3861 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:55.100010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4dca-0c14-4a82-98a1-9aca7a3e3861 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:55.100010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4dca-0c14-4a82-98a1-9aca7a3e3861 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:55.101352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 685/1364] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:55.106738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222297-1e11-43a2-adae-8783a672c7df None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:55.107194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222297-1e11-43a2-adae-8783a672c7df None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:55.107294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222297-1e11-43a2-adae-8783a672c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:55.124275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:55.179677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5f222297-1e11-43a2-adae-8783a672c7df None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:55.180915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f222297-1e11-43a2-adae-8783a672c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:55.295075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 680/1365] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:55.587175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:55.587493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:55.587773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:55.590070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 686/1366] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 21:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:55.594943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b5fe-e3f4-4b2a-9c17-5ca1905b4ae8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:55.595335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b5fe-e3f4-4b2a-9c17-5ca1905b4ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:55.595688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b5fe-e3f4-4b2a-9c17-5ca1905b4ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:55.679570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-63e1b5fe-e3f4-4b2a-9c17-5ca1905b4ae8 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:55.681289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-63e1b5fe-e3f4-4b2a-9c17-5ca1905b4ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:55.727255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:55.814881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 681/1367] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 21:47:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:55.822723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:55.824503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:55.995280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0202603-343f-47a2-9711-cceac4621ff6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:56.018233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:56.121071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:56.121737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:56.122069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:56.122259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:56.123774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:56.226606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:56.228228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a636eca-ceb9-4379-a523-5830ad18b535 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:56.248243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 687/1368] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:56.813477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723716e3-5b42-4e7e-b80d-5afa5dfbb862 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:56.814035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723716e3-5b42-4e7e-b80d-5afa5dfbb862 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:56.814148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723716e3-5b42-4e7e-b80d-5afa5dfbb862 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:56.815661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 682/1369] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:56.821292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408073e-c0b2-41c4-b5dd-5c22a5d9224e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:56.821908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408073e-c0b2-41c4-b5dd-5c22a5d9224e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:56.822473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408073e-c0b2-41c4-b5dd-5c22a5d9224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:56.856974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:56.895284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1408073e-c0b2-41c4-b5dd-5c22a5d9224e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:56.896586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1408073e-c0b2-41c4-b5dd-5c22a5d9224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:57.001513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 688/1370] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:57.617549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:57.617936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:57.618219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:57.619809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 683/1371] 10.222.0.82 () {54 vars in 964 bytes} [Thu Aug 18 21:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:57.624804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d543d-0fc8-45f4-8b6f-0f807fd9c76c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:57.625296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d543d-0fc8-45f4-8b6f-0f807fd9c76c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:57.625544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d543d-0fc8-45f4-8b6f-0f807fd9c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:57.706949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fa5d543d-0fc8-45f4-8b6f-0f807fd9c76c None None] MFA Rules not processed for user `c9b6922a858f459cbbaf1d5cad9967d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:57.708302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa5d543d-0fc8-45f4-8b6f-0f807fd9c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:57.756688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:57.788921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 689/1372] 10.222.0.82 () {58 vars in 1061 bytes} [Thu Aug 18 21:47:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:57.796341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:57.798000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:57.908512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe29b9-3cab-4bcd-9783-ae2817cf22cc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:57.951971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:57.999450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:58.000448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:58.000832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:58.000988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:58.002626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:58.141715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:58.143593 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70c4d6-b2a6-4f62-abe1-1f19936ef159 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:58.174174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 684/1373] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:47:58.282459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:58.282754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:58.283281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:58.289336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 690/1374] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:47:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:47:58.291371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae75d3-c02f-4520-ac70-aa14ec125dbd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:58.292634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae75d3-c02f-4520-ac70-aa14ec125dbd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:58.292634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae75d3-c02f-4520-ac70-aa14ec125dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:58.363768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8bae75d3-c02f-4520-ac70-aa14ec125dbd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:47:58.365071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8bae75d3-c02f-4520-ac70-aa14ec125dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:58.385944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:58.463066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 685/1375] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:47:58.752831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:47:58.754398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:58.905565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe984a9b-f82a-4e78-83bf-8515fe12e22a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:47:58.958909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:47:59.025707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:47:59.026319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:47:59.026601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:47:59.026870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:47:59.028470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:47:59.121178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:47:59.122829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bf0ef-8021-4912-84d1-4bbd88324dec service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:47:59.142202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 691/1376] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:48:01.076693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:01.077226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:01.077226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:01.078637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 686/1377] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:01.083842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581cbd-3168-406e-9c9b-b358628dd614 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:01.084288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581cbd-3168-406e-9c9b-b358628dd614 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:01.084395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581cbd-3168-406e-9c9b-b358628dd614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:01.156003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-64581cbd-3168-406e-9c9b-b358628dd614 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:01.157694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-64581cbd-3168-406e-9c9b-b358628dd614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:01.166136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:01.283578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 692/1378] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:01.586600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:01.588052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:01.781224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403f25c-c6d7-4331-9dec-f29a5ce451bf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:01.785126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:01.905530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:01.906092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:01.906374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:01.906647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:01.908515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:02.007709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:02.009660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c237-db1e-4e39-8689-26ef5790a911 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:02.030217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 687/1379] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:03.999601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:04.000069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:04.000302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:04.001704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 693/1380] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:04.007333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88938c2-ee67-4951-8c43-0236c52fa4a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:04.007746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88938c2-ee67-4951-8c43-0236c52fa4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:04.007992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88938c2-ee67-4951-8c43-0236c52fa4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:04.020938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:04.084478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b88938c2-ee67-4951-8c43-0236c52fa4a1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:04.086312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b88938c2-ee67-4951-8c43-0236c52fa4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:04.194534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 688/1381] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:04.488130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:04.490017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:04.648806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:04.661097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df249fe5-5f0d-46b1-a86b-5990fdd25da3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:04.796469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:04.797269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:04.797694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:04.798066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:04.799804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:04.896824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:04.899141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed112-5646-4bc6-aaa4-96d7f7338d7c service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:04.910774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:04.920629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 694/1382] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:48:06.867153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:06.867761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:06.867761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:06.869067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 689/1383] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:06.874371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc174-2547-4466-85e8-bda714dbf74d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:06.874818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc174-2547-4466-85e8-bda714dbf74d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:06.874916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbc174-2547-4466-85e8-bda714dbf74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:06.951540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7afbc174-2547-4466-85e8-bda714dbf74d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:06.952783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7afbc174-2547-4466-85e8-bda714dbf74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:07.052694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 695/1384] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:07.418022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:07.419610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:07.507406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:07.573570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab88fe-60bd-4152-800b-e06a4fd45023 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:07.704916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:07.705378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:07.705687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:07.705959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:07.707580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:07.763028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:07.806011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:07.807825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeb23e-18f9-433a-bbca-aa6755192b7c service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:07.828425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 690/1385] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:10.184489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:10.185371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:10.185371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:10.186653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 696/1386] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:10.192861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956706-8edc-4d7f-a526-00491f78496c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:10.193519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956706-8edc-4d7f-a526-00491f78496c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:10.193519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956706-8edc-4d7f-a526-00491f78496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:10.266821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3d956706-8edc-4d7f-a526-00491f78496c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:10.268179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3d956706-8edc-4d7f-a526-00491f78496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:10.375206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 691/1387] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:10.731861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:10.734558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:10.820139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:10.893663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa78972-c69f-4d48-83d2-f6d0f8670cbc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:11.021857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:11.023579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:11.023956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:11.024271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:11.026355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:11.071986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:11.134096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:11.135693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d7cf-d568-4710-9268-9c69b9bb91f2 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:11.154955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 697/1388] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:13.110757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd65880-37cf-4d2e-86cb-ee653d89489e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.111310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd65880-37cf-4d2e-86cb-ee653d89489e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.111310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd65880-37cf-4d2e-86cb-ee653d89489e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.112613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 692/1389] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:13.117980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.118399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.118538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.189494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:13.190732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:13.264706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f049ed-6d0f-4581-8196-8cae6a8d7227 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.264999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f049ed-6d0f-4581-8196-8cae6a8d7227 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.265275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f049ed-6d0f-4581-8196-8cae6a8d7227 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.266657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 693/1390] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:48:13.271642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37456dc5-aa8e-4b06-9ad0-433bf4832a04 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.271924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37456dc5-aa8e-4b06-9ad0-433bf4832a04 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.272829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37456dc5-aa8e-4b06-9ad0-433bf4832a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.289264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 698/1391] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:13.311122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:13.343941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-37456dc5-aa8e-4b06-9ad0-433bf4832a04 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:13.345141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-37456dc5-aa8e-4b06-9ad0-433bf4832a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:13.443783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 694/1392] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:13.576810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:13.578043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:13.592324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:13.693190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3f137-5aa7-427d-9da2-85bebe392cb0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:13.747352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0afe46-0afc-4c87-a93f-f5d71aca7525 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.747624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0afe46-0afc-4c87-a93f-f5d71aca7525 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.747901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0afe46-0afc-4c87-a93f-f5d71aca7525 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.749251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 695/1393] 10.222.0.82 () {54 vars in 967 bytes} [Thu Aug 18 21:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:48:13.754101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.754392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.754672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.815013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:13.815721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:13.816011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:13.816289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:13.817959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:13.827748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:13.834052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] MFA Rules not processed for user `71ebfbeddd1943f6af2c0229edbbc41d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:13.835368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:13.885298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:13.912079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:13.914504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 696/1394] 10.222.0.82 () {58 vars in 1064 bytes} [Thu Aug 18 21:48:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:13.914653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d32eb1-62d4-47cb-840e-5e2e68309839 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:13.922083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:13.930393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888342c2-3cd4-42df-bcfa-46b420b92309 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:13.933130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 699/1395] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:14.010083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:14.010717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:14.010998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:14.011272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:14.013032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:14.136898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:14.138402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66d4f4-96d1-44d7-8503-12e86c25483a service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:14.153552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:14.160732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 697/1396] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:48:14.911961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7217-592c-4d7e-986f-07401bdc31a7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:14.912439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7217-592c-4d7e-986f-07401bdc31a7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:14.912551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7217-592c-4d7e-986f-07401bdc31a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:14.983001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c09b7217-592c-4d7e-986f-07401bdc31a7 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:14.984193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c09b7217-592c-4d7e-986f-07401bdc31a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:15.070954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:15.075361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 700/1397] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:15.117512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:15.117849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:15.118129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:15.119503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 698/1398] 10.222.0.82 () {54 vars in 943 bytes} [Thu Aug 18 21:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:15.125868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15bed2-e1b8-4f9c-9395-4c64f2e0c1b2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:15.126158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15bed2-e1b8-4f9c-9395-4c64f2e0c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:15.126436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15bed2-e1b8-4f9c-9395-4c64f2e0c1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:15.204399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4c15bed2-e1b8-4f9c-9395-4c64f2e0c1b2 None None] MFA Rules not processed for user `156f961161644bc6a2f19e0b1a0a32c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:15.205515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4c15bed2-e1b8-4f9c-9395-4c64f2e0c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:15.315458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 701/1399] 10.222.0.82 () {58 vars in 1040 bytes} [Thu Aug 18 21:48:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:15.324800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:15.326380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:15.466714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148ecec-fa46-4688-83ed-37d496b42114 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:15.552397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:15.573628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:15.574267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:15.574541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:15.575153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:15.576432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:15.662186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:15.663817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361ecc8-bff5-471d-a8d2-d66c8f331917 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:15.672231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 699/1400] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:48:15.876057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:15.876376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:15.876782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:15.878299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 702/1401] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:15.884517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114c619-a51c-445a-abe3-44742c11bf47 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:15.884803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114c619-a51c-445a-abe3-44742c11bf47 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:15.885077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114c619-a51c-445a-abe3-44742c11bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:15.962226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4114c619-a51c-445a-abe3-44742c11bf47 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:15.963621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4114c619-a51c-445a-abe3-44742c11bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:16.043484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:16.078942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 700/1402] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:16.367770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:16.369317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:16.439939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:16.549371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd14a7-627e-4a09-a885-38e13e9ba080 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:16.673376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:16.673937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:16.674212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:16.674480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:16.676073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:16.698217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:16.782248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:16.784201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa0224-b1a7-4c65-a626-fb32319d4a96 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:16.804927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 703/1403] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:18.788338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:18.788898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:18.788898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:18.790188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 701/1404] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:18.795362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5821-77f6-45f3-99e0-255821c93970 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:18.795817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5821-77f6-45f3-99e0-255821c93970 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:18.795922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5821-77f6-45f3-99e0-255821c93970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:18.873616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-01bf5821-77f6-45f3-99e0-255821c93970 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:18.875367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-01bf5821-77f6-45f3-99e0-255821c93970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:18.984176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:19.004861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 704/1405] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:19.310751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:19.312265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:19.500076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baefff3-5ccc-4b77-bbc5-a0be38f9b066 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:19.594434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:19.672153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:19.673544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:19.673544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:19.673544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:19.675221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:19.786088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:19.788682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3881d8-591d-4d42-8c6c-4181c0b9c31b service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:19.808148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 702/1406] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:21.781991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:21.782566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:21.782566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:21.790021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2fbb-70f1-483f-a5c1-0562da726cf9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:21.790606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2fbb-70f1-483f-a5c1-0562da726cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:21.790606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2fbb-70f1-483f-a5c1-0562da726cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:21.795456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 705/1407] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:48:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:21.865672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1fea2fbb-70f1-483f-a5c1-0562da726cf9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:21.867418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1fea2fbb-70f1-483f-a5c1-0562da726cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:21.880590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:21.976721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 703/1408] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:22.268202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:22.270071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:22.438790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7395-65cd-4325-9204-0800d9c50e02 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:22.513893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:22.572404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:22.573091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:22.573371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:22.573674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:22.575490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:22.726092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:22.728288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8590f1-28c9-499c-9a33-7442474e6b78 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:22.752679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 706/1409] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:31.974522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=38 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:32.749540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=64 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:37.712145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d001-e6ed-43c4-ab74-3a6db0261e1c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:37.712719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d001-e6ed-43c4-ab74-3a6db0261e1c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:37.712719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d001-e6ed-43c4-ab74-3a6db0261e1c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:37.714089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 704/1410] 10.222.0.82 () {54 vars in 918 bytes} [Thu Aug 18 21:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:37.719667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15a499-55b4-43eb-9ca4-f470ece681d0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:37.720061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15a499-55b4-43eb-9ca4-f470ece681d0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:37.720222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15a499-55b4-43eb-9ca4-f470ece681d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:37.794730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0c15a499-55b4-43eb-9ca4-f470ece681d0 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:37.795981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c15a499-55b4-43eb-9ca4-f470ece681d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:37.900797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 707/1411] 10.222.0.82 () {58 vars in 1015 bytes} [Thu Aug 18 21:48:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:37.915522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877caef-19c4-4502-a556-f91fd6dd42de None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:37.915807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877caef-19c4-4502-a556-f91fd6dd42de None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:37.916079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877caef-19c4-4502-a556-f91fd6dd42de None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:37.917451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 705/1412] 10.222.0.82 () {54 vars in 918 bytes} [Thu Aug 18 21:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:37.922090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b472459-e0dd-4410-88c4-db3fc5afdd2a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:37.922378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b472459-e0dd-4410-88c4-db3fc5afdd2a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:37.922652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b472459-e0dd-4410-88c4-db3fc5afdd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:37.995073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:37.997827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7b472459-e0dd-4410-88c4-db3fc5afdd2a None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:37.999668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7b472459-e0dd-4410-88c4-db3fc5afdd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:38.116659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 708/1413] 10.222.0.82 () {58 vars in 1015 bytes} [Thu Aug 18 21:48:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:40.477178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:40.477756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:40.477756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:40.479084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:40.484336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4b8f-75c7-4d4c-b470-8b0a5b4182d9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:40.484756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4b8f-75c7-4d4c-b470-8b0a5b4182d9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:40.484894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4b8f-75c7-4d4c-b470-8b0a5b4182d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:40.550840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-32be4b8f-75c7-4d4c-b470-8b0a5b4182d9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:40.552047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-32be4b8f-75c7-4d4c-b470-8b0a5b4182d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:40.607678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:40.651559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:40.937717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:40.939317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:41.087827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d08c-6300-45ec-bbf2-3d9d7577dbf1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:41.159205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:41.200063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:41.200597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:41.200878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:41.201152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:41.202800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:41.301953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:41.303688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf7ce6-bc49-4d6e-850a-6510ed506a1a service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:41.322086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 707/1416] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:42.212321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:42.212927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:42.212927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:42.214237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:42.219184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357032-bdb5-4a30-99c8-f48fb39074d5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:42.219594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357032-bdb5-4a30-99c8-f48fb39074d5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:42.219741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357032-bdb5-4a30-99c8-f48fb39074d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:42.294608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-34357032-bdb5-4a30-99c8-f48fb39074d5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:42.295842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-34357032-bdb5-4a30-99c8-f48fb39074d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:42.322879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:42.392233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:48:42.544015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:42.545665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:42.691682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d09a5a-d75b-4c11-b3d0-eb7d00bbca77 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:42.732167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:42.805023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:42.805557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:42.805862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:42.806131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:42.807722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:42.895792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:42.897952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef05e44-8728-4b33-a366-1c43c0d10ef2 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:42.916806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 711/1419] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:43.812930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:43.813478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:43.813478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:43.814856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:43.820397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a2ed1-59dc-4d40-aca9-257583aaca9d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:43.820832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a2ed1-59dc-4d40-aca9-257583aaca9d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:43.820947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a2ed1-59dc-4d40-aca9-257583aaca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:43.876058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:43.898472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-952a2ed1-59dc-4d40-aca9-257583aaca9d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:43.899912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-952a2ed1-59dc-4d40-aca9-257583aaca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:43.994354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:44.148027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:44.149542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:44.295816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fb967-3f34-483b-bc39-a3c2bdc8d286 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:44.308301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:44.407893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:44.408508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:44.408787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:44.409307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:44.410935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:44.619438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:44.620961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea615e2-01e3-475e-bf1d-fe4f4b63fa34 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:44.638531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 710/1422] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:45.501831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:45.502381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:45.502381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:45.503736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:45.508852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada9303-3701-4464-98d5-de0755a52e6d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:45.509281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada9303-3701-4464-98d5-de0755a52e6d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:45.509377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada9303-3701-4464-98d5-de0755a52e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:45.529203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:45.585350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2ada9303-3701-4464-98d5-de0755a52e6d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:45.586598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ada9303-3701-4464-98d5-de0755a52e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:45.683424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:45.832611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:45.834679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:45.944727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:45.980264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea290-0b73-434d-bfee-7ef572856c60 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:46.091159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:46.091693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:46.091973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:46.092252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:46.093902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:46.169202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:46.182551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:46.184177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233b998-4381-4293-88ea-ea6bac68e1f7 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:46.202870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 714/1425] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:47.059404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:47.059980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:47.059980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:47.061261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:47.066197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd8a9-ef0f-4e9a-b86b-b3559b9e9d3e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:47.066607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd8a9-ef0f-4e9a-b86b-b3559b9e9d3e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:47.066757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd8a9-ef0f-4e9a-b86b-b3559b9e9d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:47.143001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c08fd8a9-ef0f-4e9a-b86b-b3559b9e9d3e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:47.144209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c08fd8a9-ef0f-4e9a-b86b-b3559b9e9d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:47.240585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:47.392123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:47.393709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:47.479053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:47.540261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624ea9-3f8f-419a-bfe7-df9c00209b8d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:47.652301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:47.652816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:47.653090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:47.653357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:47.654999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:47.703250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:47.745672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:47.747287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254622a-73e0-489e-9ffa-8de2fd3cb228 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:47.764777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 713/1428] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:48.571535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:48.572050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:48.572157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:48.573464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:48.578505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77358-e86f-4da7-9ae2-71b4842ec8df None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:48.578921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77358-e86f-4da7-9ae2-71b4842ec8df None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:48.579044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77358-e86f-4da7-9ae2-71b4842ec8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:48.654466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ff77358-e86f-4da7-9ae2-71b4842ec8df None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:48.655674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ff77358-e86f-4da7-9ae2-71b4842ec8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:48.751967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:48.903527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:48.904956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:48.953381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:49.051381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356da9a1-9b92-4a8b-ab4e-e998fa078272 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:49.167068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:49.167604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:49.167888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:49.168167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:49.169857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:49.176841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:49.261843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:49.263427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f0198-2e74-47f7-a4fe-d417b99d5771 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:49.280894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 717/1431] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:50.086429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:50.087004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:50.087004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:50.088320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:50.093445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6420bad-b506-4ff3-ae4f-5d311665878c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:50.093885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6420bad-b506-4ff3-ae4f-5d311665878c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:50.094032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6420bad-b506-4ff3-ae4f-5d311665878c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:50.172332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b6420bad-b506-4ff3-ae4f-5d311665878c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:50.173577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6420bad-b506-4ff3-ae4f-5d311665878c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:50.240783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:50.270193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:50.422439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:50.423886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:50.441800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:50.569180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde1d44-80af-4dc1-9a7f-b39eae0ccf2d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:50.654323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:50.683516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:50.684051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:50.684323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:50.684589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:50.686205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:50.773978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:50.775561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340e6df-1a98-4400-8d83-ffc60f2969e4 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:50.794786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 716/1434] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:51.593589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:51.594172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:51.594172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:51.595508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:51.600498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f63703-01e9-4f0e-8782-e7da7b698bcb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:51.600918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f63703-01e9-4f0e-8782-e7da7b698bcb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:51.601045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f63703-01e9-4f0e-8782-e7da7b698bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:51.674963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b4f63703-01e9-4f0e-8782-e7da7b698bcb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:51.676169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4f63703-01e9-4f0e-8782-e7da7b698bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:51.721111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:51.778259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:51.927415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:51.929373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:52.083112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9f881-5c9c-4125-a0d1-7fe52c452fa5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:52.146370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:52.210741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:52.211550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:52.211832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:52.212106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:52.213758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:52.310576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:52.312204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12cfc-7b39-4f16-86e4-4da164caa04b service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:52.330834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 720/1437] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:53.140968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:53.141528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:53.141528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:53.142900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:53.149183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07b66-d901-42a9-b4f7-3ae21894c911 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:53.149622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07b66-d901-42a9-b4f7-3ae21894c911 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:53.149774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff07b66-d901-42a9-b4f7-3ae21894c911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:53.224603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7ff07b66-d901-42a9-b4f7-3ae21894c911 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:53.225873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ff07b66-d901-42a9-b4f7-3ae21894c911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:53.227894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:53.322623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:53.475644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:53.477154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:53.618922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837d685-c096-4429-b98b-a00745117ed1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:53.646478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:53.735005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:53.735619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:53.735901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:53.736173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:53.737822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:53.826006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:53.827627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b8ac9-c78e-4de8-907d-3fd5e957f5e6 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:53.845001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 719/1440] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:48:55.672448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:55.673020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:55.673020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:55.674376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:55.679662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5d72-ff8b-42ca-afb4-ddd55e6870f2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:55.680085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5d72-ff8b-42ca-afb4-ddd55e6870f2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:55.680195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5d72-ff8b-42ca-afb4-ddd55e6870f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:55.721552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:55.747049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-117b5d72-ff8b-42ca-afb4-ddd55e6870f2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:55.748240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-117b5d72-ff8b-42ca-afb4-ddd55e6870f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:55.842724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:56.124395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:56.125981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:56.228712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea7bc5-bf1d-4120-b6c5-484012806d02 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:56.260671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:56.302625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:56.303269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:56.303545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:56.303811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:56.305411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:56.418769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:56.420128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dabed-f0b7-4326-a507-9bbd33efd4dd service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:56.440223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 723/1443] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:48:58.477146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:58.477776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:58.477776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:58.479044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:48:58.484209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3289a-cdd6-4351-b72d-8e69270ee794 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:58.484602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3289a-cdd6-4351-b72d-8e69270ee794 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:58.484792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3289a-cdd6-4351-b72d-8e69270ee794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:58.550444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ffb3289a-cdd6-4351-b72d-8e69270ee794 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:48:58.551062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ffb3289a-cdd6-4351-b72d-8e69270ee794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:58.564958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:58.650347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:48:58.935771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:48:58.937306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:59.042318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a148f86-89da-4fad-8c04-f07f06845310 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:48:59.054371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:48:59.116020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:48:59.116632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:48:59.116912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:48:59.117187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:48:59.118841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:48:59.230385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:48:59.231743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaaba5-8eb9-459a-a9de-0a6f743884e4 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:48:59.249330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 722/1446] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:01.499778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:01.500323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:01.500323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:01.501699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:01.506958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f451e-96f4-4706-9d3f-7e642f781e1c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:01.507352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f451e-96f4-4706-9d3f-7e642f781e1c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:01.507517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f451e-96f4-4706-9d3f-7e642f781e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:01.557026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:01.574545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fe9f451e-96f4-4706-9d3f-7e642f781e1c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:01.575756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fe9f451e-96f4-4706-9d3f-7e642f781e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:01.670423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:01.952810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:01.954443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:02.057346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f40dbf-0d69-434a-8ccd-5d255984174f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:02.098421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:02.132560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:02.133175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:02.133458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:02.133769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:02.135387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:02.241906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:02.243267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55767a58-f016-43f6-a83a-e7dc94cd187b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:02.261563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 726/1449] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:04.303416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:04.304007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:04.304007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:04.305301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:04.310497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3606-22f2-486c-942c-b41545d5664d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:04.310934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3606-22f2-486c-942c-b41545d5664d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:04.311074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3606-22f2-486c-942c-b41545d5664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:04.377279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-388c3606-22f2-486c-942c-b41545d5664d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:04.378527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-388c3606-22f2-486c-942c-b41545d5664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:04.398793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:04.475752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:04.763000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:04.764572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:04.866638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489556-4f39-4b4c-8727-bf066de243cf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:04.886337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:04.939786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:04.940386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:04.940659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:04.940926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:04.942551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:05.049471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:05.050933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a587add-d025-41a4-b51f-e7064d42fb7a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:05.070401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 725/1452] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:07.015220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:07.015745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:07.015745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:07.017027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:07.022881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb7adf-32f5-455b-97a0-c41d4a389a83 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:07.023331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb7adf-32f5-455b-97a0-c41d4a389a83 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:07.023430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb7adf-32f5-455b-97a0-c41d4a389a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:07.086139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:07.089708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-68fb7adf-32f5-455b-97a0-c41d4a389a83 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:07.091331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68fb7adf-32f5-455b-97a0-c41d4a389a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:07.188302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:07.469087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:07.470671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:07.573295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441868c-a2d6-48ab-9b2f-241ff11ab700 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:07.621915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:07.647363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:07.647992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:07.648277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:07.648549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:07.650331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:07.761153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:07.762661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0383bd-6028-4f66-ae12-62b674cb4659 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:07.780890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 729/1455] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:09.769996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:09.770550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:09.770550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:09.772001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:09.777148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48bc7-75f8-4afc-8b57-c8d503534b7d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:09.777791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48bc7-75f8-4afc-8b57-c8d503534b7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:09.777791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c48bc7-75f8-4afc-8b57-c8d503534b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:09.843375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-72c48bc7-75f8-4afc-8b57-c8d503534b7d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:09.844586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-72c48bc7-75f8-4afc-8b57-c8d503534b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:09.874516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:09.943521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:10.226968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:10.228521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:10.331643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cce506-a31f-41d3-860d-225cd5f80f38 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:10.360252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:10.404873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:10.405475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:10.405781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:10.406046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:10.407642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:10.515049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:10.516519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff56c-5918-442f-a528-26dad7856e24 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:10.536090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 728/1458] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:12.517119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:12.517714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:12.517714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:12.519059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:49:12.525134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a0fb-b7fc-4baa-863e-6b9d5ec3ad30 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:12.525541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a0fb-b7fc-4baa-863e-6b9d5ec3ad30 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:12.525743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a0fb-b7fc-4baa-863e-6b9d5ec3ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:12.590824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c249a0fb-b7fc-4baa-863e-6b9d5ec3ad30 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:12.592023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c249a0fb-b7fc-4baa-863e-6b9d5ec3ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:12.600033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:12.687536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:12.973919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:12.975509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:13.078484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20608-9dc4-4e44-ba44-3d45fd1da88c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:13.133236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:13.151437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:13.152075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:13.152356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:13.152631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:13.154291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:13.262577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:13.263937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397643ab-a17d-4828-b9ce-375df94b7e9c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:13.282074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 732/1461] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:15.218253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:15.218881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:15.218881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:15.220261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:15.226312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab288ad-c7c1-457d-8c94-8673445d95d0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:15.226731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab288ad-c7c1-457d-8c94-8673445d95d0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:15.226887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab288ad-c7c1-457d-8c94-8673445d95d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:15.293094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1ab288ad-c7c1-457d-8c94-8673445d95d0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:15.294344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1ab288ad-c7c1-457d-8c94-8673445d95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:15.331793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:15.390114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:15.673619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:15.675198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:15.778749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc016b-617d-453b-bba4-a8af50624dc9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:15.815249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:15.851008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:15.851611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:15.851889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:15.852159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:15.853788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:15.962712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:15.964058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f78b87-d7c4-449c-9812-f56e2a1ccfb3 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:15.982522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 731/1464] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:17.924842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:17.925412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:17.925412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:17.926784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:17.931988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf74c77-4e23-485c-b7fb-9c81dd0e9e2d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:17.932414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf74c77-4e23-485c-b7fb-9c81dd0e9e2d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:17.932568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf74c77-4e23-485c-b7fb-9c81dd0e9e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:17.998362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-adf74c77-4e23-485c-b7fb-9c81dd0e9e2d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:17.999480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-adf74c77-4e23-485c-b7fb-9c81dd0e9e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:18.017587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:18.094555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:18.377787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:18.379407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:18.484284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad0f0-647c-431d-82ac-53e0a6e53b4a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:18.550013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:18.557878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:18.558489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:18.558767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:18.559038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:18.560640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:18.668607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:18.670007 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d767-fa8f-48d3-bcb6-5f52405d5454 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:18.688336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 735/1467] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:20.605629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:20.606189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:20.606189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:20.607550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:20.612799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022e768-1935-4550-8e86-91c65eebef99 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:20.613233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022e768-1935-4550-8e86-91c65eebef99 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:20.613341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022e768-1935-4550-8e86-91c65eebef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:20.681479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0022e768-1935-4550-8e86-91c65eebef99 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:20.682729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0022e768-1935-4550-8e86-91c65eebef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:20.731399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:20.781871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:21.061683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:21.063222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:21.166341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdaa59-f8d5-4603-9844-8e41ec7d7886 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:21.214818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:21.242276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:21.242890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:21.243166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:21.243435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:21.245102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:21.353873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:21.355244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e83d-8795-4c08-a272-0f91161c54e5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:21.374142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 734/1470] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:30.780035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=22 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:31.371893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=66 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:40.738575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7e5a-1c76-4004-9bd6-a02dfd694d4d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:40.739046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7e5a-1c76-4004-9bd6-a02dfd694d4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:40.739142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7e5a-1c76-4004-9bd6-a02dfd694d4d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:40.740508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 737/1471] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:40.745856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed4e0-1f31-4cdf-af28-6fb9865757ba None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:40.746292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed4e0-1f31-4cdf-af28-6fb9865757ba None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:40.746404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ed4e0-1f31-4cdf-af28-6fb9865757ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:40.812208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5b6ed4e0-1f31-4cdf-af28-6fb9865757ba None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:40.813405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b6ed4e0-1f31-4cdf-af28-6fb9865757ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:40.906458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 735/1472] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:41.204001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:41.204285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:41.204555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:41.205901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 738/1473] 10.222.0.82 () {54 vars in 967 bytes} [Thu Aug 18 21:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:41.211413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debce135-7d60-4c7b-8fbd-7e7248d990e8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:41.211701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debce135-7d60-4c7b-8fbd-7e7248d990e8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:41.211976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debce135-7d60-4c7b-8fbd-7e7248d990e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:41.287130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-debce135-7d60-4c7b-8fbd-7e7248d990e8 None None] MFA Rules not processed for user `71ebfbeddd1943f6af2c0229edbbc41d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:41.288896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-debce135-7d60-4c7b-8fbd-7e7248d990e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:41.308925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:41.363876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 736/1474] 10.222.0.82 () {58 vars in 1064 bytes} [Thu Aug 18 21:49:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:41.372066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:41.373591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:41.475549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff810fe0-d1d8-4153-8124-dfc3341dfe1a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:41.550491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:41.551103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:41.551383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:41.551655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:41.553914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:41.728524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:41.783984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:41.785722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50adf5c-a081-4f16-8dd2-549b5d9e66a3 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:41.804549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 739/1475] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:43.789628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:43.790211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:43.790211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:43.791520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 737/1476] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:43.796789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab7fae-a7c0-4639-8e8c-92b33ce38751 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:43.797195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab7fae-a7c0-4639-8e8c-92b33ce38751 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:43.797345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab7fae-a7c0-4639-8e8c-92b33ce38751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:43.862011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e4ab7fae-a7c0-4639-8e8c-92b33ce38751 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:43.863205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e4ab7fae-a7c0-4639-8e8c-92b33ce38751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:43.956508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 740/1477] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:44.228809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:44.230804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:44.362283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734590ba-9abe-4ba6-907e-01bb4290fcc3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:44.410370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:44.454864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:44.455474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:44.455750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:44.456019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:44.456651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:44.458899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:49:44.458899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:49:44.458899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:49:44.460053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4b06-9063-40dc-bed5-45b39d88d7c4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:44.480167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 738/1478] 10.222.0.82 () {56 vars in 1108 bytes} [Thu Aug 18 21:49:44 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:49:46.401130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:46.401721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:46.401721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:46.403076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 741/1479] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:46.408376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a546189-4177-40c0-94a2-84e31b56cd13 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:46.408801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a546189-4177-40c0-94a2-84e31b56cd13 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:46.408910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a546189-4177-40c0-94a2-84e31b56cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:46.474612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6a546189-4177-40c0-94a2-84e31b56cd13 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:46.475842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6a546189-4177-40c0-94a2-84e31b56cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:46.572682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 739/1480] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:49:46.863791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:46.866250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:46.875814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:47.012378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af0f9-89fa-409a-9e22-89284ebeba0e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:47.092336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:47.126438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:47.127049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:47.127329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:47.127601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:47.129201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:47.216282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:47.217891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f371e-37c0-495a-8afd-0f847b17df32 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:47.235884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 742/1481] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:49.316678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:49.317223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:49.317223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:49.319120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 740/1482] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:49.324316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d14d2a-2161-496f-924f-b6b7ff0c2f93 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:49.324752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d14d2a-2161-496f-924f-b6b7ff0c2f93 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:49.324872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d14d2a-2161-496f-924f-b6b7ff0c2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:49.389450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-36d14d2a-2161-496f-924f-b6b7ff0c2f93 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:49.390708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36d14d2a-2161-496f-924f-b6b7ff0c2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:49.435748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:49.488351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 743/1483] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:49.778923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:49.780528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:49.804373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:49.884663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3aad-1ba4-4a44-a901-d3df61656da6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:49.956865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:49.957478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:49.957784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:49.958057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:49.959647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:50.031489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:50.066963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:50.068439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cdeec-383f-4b9d-b2de-f9622954a3d2 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:50.086506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 741/1484] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:52.030270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:52.030860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:52.030860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:52.032146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 744/1485] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:52.038106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55cbe-c839-4c48-99c2-46fbe8c762d3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:52.038567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55cbe-c839-4c48-99c2-46fbe8c762d3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:52.038666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55cbe-c839-4c48-99c2-46fbe8c762d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:52.111760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0fb55cbe-c839-4c48-99c2-46fbe8c762d3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:52.112977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0fb55cbe-c839-4c48-99c2-46fbe8c762d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:52.209496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 742/1486] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:52.499813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:52.501361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:52.604773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81a04-bc59-40ea-9582-d425996ef648 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:52.678214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:52.679987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:52.680634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:52.680914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:52.681316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:52.683622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:52.792403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:52.793860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644328e-2cbf-4357-85be-85d63550f8ac service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:52.812036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 745/1487] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:54.789452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:54.790095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:54.790095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:54.791415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 743/1488] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:54.796577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8a2f-9cd3-4538-8d93-8cb8378bc6e3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:54.796959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8a2f-9cd3-4538-8d93-8cb8378bc6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:54.797121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8a2f-9cd3-4538-8d93-8cb8378bc6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:54.863793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-368e8a2f-9cd3-4538-8d93-8cb8378bc6e3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:54.865560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-368e8a2f-9cd3-4538-8d93-8cb8378bc6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:54.918398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:54.961821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 746/1489] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:55.251231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:55.252799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:55.289069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:55.358869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6acb2-6109-4211-bf21-6425fabe9b31 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:55.431958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:55.432564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:55.432847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:55.433118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:55.434763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:55.512604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:55.541761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:55.543197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0009-f607-403c-805d-aac5ee28fb64 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:55.561819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 744/1490] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:49:57.628849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:57.629419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:57.629419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:57.631453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 747/1491] 10.222.0.82 () {54 vars in 927 bytes} [Thu Aug 18 21:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:49:57.636967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2eef5-3f64-4fee-ba43-56058ef51d4a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:57.637398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2eef5-3f64-4fee-ba43-56058ef51d4a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:57.637542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2eef5-3f64-4fee-ba43-56058ef51d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:57.706546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a0e2eef5-3f64-4fee-ba43-56058ef51d4a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:49:57.707769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a0e2eef5-3f64-4fee-ba43-56058ef51d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:57.803381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 745/1492] 10.222.0.82 () {58 vars in 1024 bytes} [Thu Aug 18 21:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:49:58.090035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:49:58.091590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:58.195748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ba17b-f6d5-45d1-9f08-5164c5ada61c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:49:58.267987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:49:58.268594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:49:58.268875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:49:58.269145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:49:58.270813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:49:58.279765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:49:58.386129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:49:58.387481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed38b4-1cf5-4eef-965e-06c1deb4107b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:49:58.407178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 748/1493] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:50:07.801510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=82 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:50:08.405032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=50 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:17.062781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:17.063388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:17.063388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:17.138095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:51:17.139289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:17.234773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 746/1494] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:51:17.243486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:17.243994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:17.244164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:17.320727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:51:17.321898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:17.419529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 749/1495] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:51:17.424211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:17.425428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:17.436873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573072-c559-4d84-9dd6-dcb7de1ac0bd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:17.496788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:17.530786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:17.531394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:17.531676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:17.531947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:17.532587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:51:17.532888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:17.533935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:17.533935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:51:17.533935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:51:17.534837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:17.546589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 747/1496] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:51:17.554936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:17.564077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57683026-2d56-4e40-9acf-5e5504dda6cb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:17.625879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:17.659513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:17.660117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:17.660392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:17.660659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:17.662253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:17.663986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:17.663986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:51:17.663986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:51:17.665207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:17.722222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 750/1497] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:51:17 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:51:17.726433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:17.727999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:17.761029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135a1b-54c6-4305-958c-9df68150ee37 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:17.854786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:17.855384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:17.855659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:17.855925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:17.856548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:17.857546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:17.857546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:51:17.857546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:51:17.858796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:17.901480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 748/1498] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:51:17 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:51:17.905860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:17.907375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:17.969021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630828c-8349-49d9-ba54-d488af5303ae admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:18.053462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:18.065012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:18.065653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.065937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.066212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.066854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:18.068507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:18.068507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:51:18.068507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:51:18.069387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:18.081506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 751/1499] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:51:18.085617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:18.087148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:18.098488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1eda9-8f51-4dfc-8ec1-c2623b321a72 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:18.123294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:18.192126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:18.193234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.193506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.193801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.194442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:18.195442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:18.195442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:51:18.195442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:51:18.196339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b63885-e7e8-4fe6-81f0-33542b8d5018 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:18.208972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 749/1500] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:51:18.213210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:18.214779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:18.248378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ca65-c732-4e85-bd69-5fb49a9c9623 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:18.341294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:18.341981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.342260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.342537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] PATH_INFO: `/v3/projects/4b252acee2fa4edca512314348e75925/users/770a94d40a4f430c9efc424edca13bf7/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.363645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b252acee2fa4edca512314348e75925, user_id=770a94d40a4f430c9efc424edca13bf7, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:18.365255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:18.365255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:51:18.365255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:51:18.367216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a53919-e3ca-4778-ac94-9d93eb65b9de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:18.392511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 752/1501] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:51:18 2022] PUT /identity/v3/projects/4b252acee2fa4edca512314348e75925/users/770a94d40a4f430c9efc424edca13bf7/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:51:18.404705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579bffb-5b49-4555-b4f2-7b289adb03a2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.404983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579bffb-5b49-4555-b4f2-7b289adb03a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.405254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579bffb-5b49-4555-b4f2-7b289adb03a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.463316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e579bffb-5b49-4555-b4f2-7b289adb03a2 None None] MFA Rules not processed for user `770a94d40a4f430c9efc424edca13bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:51:18.464491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e579bffb-5b49-4555-b4f2-7b289adb03a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:18.541669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 750/1502] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:51:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:51:18.555657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14652e2-2128-4d68-9786-de974217b31a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.555947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14652e2-2128-4d68-9786-de974217b31a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.556224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14652e2-2128-4d68-9786-de974217b31a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.557570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 753/1503] 10.222.0.82 () {54 vars in 964 bytes} [Thu Aug 18 21:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:51:18.562603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d138-46e4-489e-a331-298b56edbfcb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.562890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d138-46e4-489e-a331-298b56edbfcb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.563166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d138-46e4-489e-a331-298b56edbfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.593571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:18.640056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f072d138-46e4-489e-a331-298b56edbfcb None None] MFA Rules not processed for user `c9b6922a858f459cbbaf1d5cad9967d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:51:18.641271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f072d138-46e4-489e-a331-298b56edbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:18.718448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 751/1504] 10.222.0.82 () {58 vars in 1061 bytes} [Thu Aug 18 21:51:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:51:18.726900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14652e2-2128-4d68-9786-de974217b31a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:18.728419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b14652e2-2128-4d68-9786-de974217b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:18.778284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:18.832103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14652e2-2128-4d68-9786-de974217b31a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:18.904198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:18.904823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:18.905084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:18.905354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:18.906984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:19.005311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:19.006682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:19.010145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:19.025741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 754/1505] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:51:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:51:19.070767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd425dc6-e71d-4ced-b9f6-f4b7a972cbc0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:19.071056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd425dc6-e71d-4ced-b9f6-f4b7a972cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:19.071332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd425dc6-e71d-4ced-b9f6-f4b7a972cbc0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:19.072600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 752/1506] 10.222.0.82 () {52 vars in 830 bytes} [Thu Aug 18 21:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:51:19.076327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:19.077568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:19.178449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a7ec-7dd6-48cb-a8cd-8e636cd6c571 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:19.273858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:19.274536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:19.274765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:19.275039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] PATH_INFO: `/v3/users/770a94d40a4f430c9efc424edca13bf7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:19.288068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770a94d40a4f430c9efc424edca13bf7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:19.289096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:19.289096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:51:19.289096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:51:19.290386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:19.299897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:19.356516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770a94d40a4f430c9efc424edca13bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:19.370871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770a94d40a4f430c9efc424edca13bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:19.392837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770a94d40a4f430c9efc424edca13bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:19.402608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770a94d40a4f430c9efc424edca13bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:19.403185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770a94d40a4f430c9efc424edca13bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:19.408862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 755/1507] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:51:19 2022] DELETE /identity/v3/users/770a94d40a4f430c9efc424edca13bf7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:51:19.414158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:19.414645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:19.414764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:19.470590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:51:19.471783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:19.546576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:19.560452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 753/1508] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:51:19.570052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:19.571274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:19.673174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b7f-9022-451e-853c-3d691cdd3ea8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:19.730657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:19.750341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:19.750956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:19.751238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:19.751511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:19.753121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:19.859449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:19.860790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9e4cc-06f0-4291-9773-8cfe77387562 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:19.878918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 756/1509] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:51:20.227779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:51:20.229075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:51:20.240640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29404d4-c810-495f-b003-9802c8bd0f2f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:51:20.337548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:51:20.338361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:51:20.338633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:51:20.338898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] PATH_INFO: `/v3/projects/4b252acee2fa4edca512314348e75925` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:51:20.348443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b252acee2fa4edca512314348e75925)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:51:20.349458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:51:20.349458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:51:20.349458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:51:20.350693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:51:20.407153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b252acee2fa4edca512314348e75925', 'request_id': 'req-9a7fce63-01d7-46a8-9a0a-8677cd63e480'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:20.413163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b252acee2fa4edca512314348e75925', 'request_id': 'req-9a7fce63-01d7-46a8-9a0a-8677cd63e480'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:20.422824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b252acee2fa4edca512314348e75925', 'request_id': 'req-9a7fce63-01d7-46a8-9a0a-8677cd63e480'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:20.453175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] The token cache is being invalidate because project 4b252acee2fa4edca512314348e75925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:51:20.453458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:20.453919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9a7fce63-01d7-46a8-9a0a-8677cd63e480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:51:20.455331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 754/1510] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:51:20 2022] DELETE /identity/v3/projects/4b252acee2fa4edca512314348e75925 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:51:29.876664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=62 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:51:30.451180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:52:49.834990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:49.835539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:49.835539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:49.836782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:52:49.841589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671f45-feaa-47c2-a996-304e2ea6e2dd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:49.842043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671f45-feaa-47c2-a996-304e2ea6e2dd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:49.842151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41671f45-feaa-47c2-a996-304e2ea6e2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:49.924628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-41671f45-feaa-47c2-a996-304e2ea6e2dd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:52:49.925866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-41671f45-feaa-47c2-a996-304e2ea6e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:52:50.029346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:52:50.182414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:52:50.184606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:52:50.300838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d4ff-f58d-4fb3-b83f-aac0c1ae8ad6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:52:50.307363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:52:50.307972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:50.308248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:50.308514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:50.316563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:52:50.318222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e93233-68ef-42b8-9a0e-0986d0703371 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:52:50.327421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 758/1513] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:52:51.160656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3e68-da2e-4d5b-b24b-9311bf557cf4 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:51.161278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3e68-da2e-4d5b-b24b-9311bf557cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:51.161278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3e68-da2e-4d5b-b24b-9311bf557cf4 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:51.162653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:52:51.167822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95fed8-2e7b-463c-9bab-4dc98bfa516e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:51.168249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95fed8-2e7b-463c-9bab-4dc98bfa516e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:51.168384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95fed8-2e7b-463c-9bab-4dc98bfa516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:51.187477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5b95fed8-2e7b-463c-9bab-4dc98bfa516e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:52:51.188689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b95fed8-2e7b-463c-9bab-4dc98bfa516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:52:51.197477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:52:51.353861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3e68-da2e-4d5b-b24b-9311bf557cf4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:52:51.368125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3e68-da2e-4d5b-b24b-9311bf557cf4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:52:51.373579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:52:51.374201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:51.374480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:51.374753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:51.383280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:52:51.384869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4008d-92b5-462b-9445-100239d6da43 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:52:51.394418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 757/1516] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:52:51.766542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8dcc-c7fa-4907-a543-205b080db5f4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:51.766829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8dcc-c7fa-4907-a543-205b080db5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:51.767110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8dcc-c7fa-4907-a543-205b080db5f4 None None] PATH_INFO: `/v3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:51.768406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 760/1517] 10.222.0.45 () {54 vars in 925 bytes} [Thu Aug 18 21:52:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:52:51.774566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e279cf-83ba-46a8-b7f4-4a194327d8b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:51.774826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e279cf-83ba-46a8-b7f4-4a194327d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:51.775163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e279cf-83ba-46a8-b7f4-4a194327d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:51.833256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a2e279cf-83ba-46a8-b7f4-4a194327d8b5 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:52:51.834506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a2e279cf-83ba-46a8-b7f4-4a194327d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:52:51.851165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:52:51.905932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 758/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 18 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:52:55.707552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fdfce-32ac-46c2-b191-5a3e9d8ad7db None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:55.708101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fdfce-32ac-46c2-b191-5a3e9d8ad7db None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:55.708101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fdfce-32ac-46c2-b191-5a3e9d8ad7db None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:55.710021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 761/1519] 10.222.0.82 () {54 vars in 961 bytes} [Thu Aug 18 21:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:52:55.715075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220729b2-268d-4c37-9dd8-de2c8f15b464 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:55.715550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220729b2-268d-4c37-9dd8-de2c8f15b464 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:55.715723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220729b2-268d-4c37-9dd8-de2c8f15b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:55.734781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-220729b2-268d-4c37-9dd8-de2c8f15b464 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:52:55.735978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-220729b2-268d-4c37-9dd8-de2c8f15b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:52:55.744349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 759/1520] 10.222.0.82 () {58 vars in 1058 bytes} [Thu Aug 18 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:52:55.752069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fdfce-32ac-46c2-b191-5a3e9d8ad7db None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:52:55.765508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fdfce-32ac-46c2-b191-5a3e9d8ad7db None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:52:55.770454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:52:55.771037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:52:55.771947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:52:55.772218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:52:55.780216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:52:55.781829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e74e4-c920-4bbb-89e4-0429cb9f3c71 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:52:55.791636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 762/1521] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:02.608435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5edf-d64b-430d-a1fc-b013952cddda None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:02.609031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5edf-d64b-430d-a1fc-b013952cddda None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:02.609031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5edf-d64b-430d-a1fc-b013952cddda None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:02.610374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:53:02.615715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0de3f-e93e-4ffb-a00c-2f0cdbcf5c55 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:02.616169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0de3f-e93e-4ffb-a00c-2f0cdbcf5c55 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:02.616277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0de3f-e93e-4ffb-a00c-2f0cdbcf5c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:02.635864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2fa0de3f-e93e-4ffb-a00c-2f0cdbcf5c55 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:02.637049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2fa0de3f-e93e-4ffb-a00c-2f0cdbcf5c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:02.644766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:53:02.798389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad5edf-d64b-430d-a1fc-b013952cddda None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:02.813337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad5edf-d64b-430d-a1fc-b013952cddda None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:02.819721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:02.820331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:02.820611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:02.820937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:02.829070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:02.830697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944551ce-cb63-4d57-a36f-f385e509ec63 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:02.840076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 761/1524] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:03.652076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d734d-6d67-4419-b783-fa2fe07bf375 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:03.652646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d734d-6d67-4419-b783-fa2fe07bf375 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:03.652646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d734d-6d67-4419-b783-fa2fe07bf375 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:03.654026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 21:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:53:03.658906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086b43f-c7f8-4a22-86c6-87b686bcd7e9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:03.659367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086b43f-c7f8-4a22-86c6-87b686bcd7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:03.659473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086b43f-c7f8-4a22-86c6-87b686bcd7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:03.680208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d086b43f-c7f8-4a22-86c6-87b686bcd7e9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:03.681479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d086b43f-c7f8-4a22-86c6-87b686bcd7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:03.689583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 21:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:53:03.841939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d734d-6d67-4419-b783-fa2fe07bf375 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:03.856954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d734d-6d67-4419-b783-fa2fe07bf375 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:03.862411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:03.863011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:03.863296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:03.863574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:03.872210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:03.873813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b723ef2-1b80-4c18-90b1-60efde86477c service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:03.883210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 765/1527] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:07.706966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:07.707526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:07.707526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:07.709491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:53:07.714805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4b84e-2aa0-4e01-9632-c1ad3bd63f76 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:07.715267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4b84e-2aa0-4e01-9632-c1ad3bd63f76 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:07.715371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4b84e-2aa0-4e01-9632-c1ad3bd63f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:07.734352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-47a4b84e-2aa0-4e01-9632-c1ad3bd63f76 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:07.735538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-47a4b84e-2aa0-4e01-9632-c1ad3bd63f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:07.743335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:53:08.028119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:08.029880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:08.108759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97559a-73d8-401a-ae52-169afd06163e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:08.112378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:08.112986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:08.113266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:08.113541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:08.122993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:08.124424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320b37e-a35b-4e0a-8087-97b89d600800 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:08.134453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 764/1530] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:10.006212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495413e-9061-4567-a24b-d9c1f708168d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:10.006740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495413e-9061-4567-a24b-d9c1f708168d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:10.006740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495413e-9061-4567-a24b-d9c1f708168d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:10.008029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:53:10.013294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290570-923a-49c3-ad6c-f8378582abbc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:10.013745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290570-923a-49c3-ad6c-f8378582abbc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:10.013931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290570-923a-49c3-ad6c-f8378582abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:10.033663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-48290570-923a-49c3-ad6c-f8378582abbc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:10.034877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48290570-923a-49c3-ad6c-f8378582abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:10.043109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:53:10.327642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495413e-9061-4567-a24b-d9c1f708168d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:10.339084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495413e-9061-4567-a24b-d9c1f708168d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:10.342583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:10.343173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:10.343453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:10.343723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:10.353280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:10.354744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb860be5-5f33-4203-9972-9d67acb12b0b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:10.363981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 768/1533] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:13.842761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:13.843322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:13.843322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:13.844698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:53:13.850458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64422b-2448-4a97-9e1a-83f75e7fce64 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:13.850898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64422b-2448-4a97-9e1a-83f75e7fce64 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:13.850995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64422b-2448-4a97-9e1a-83f75e7fce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:13.870444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4f64422b-2448-4a97-9e1a-83f75e7fce64 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:13.871641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4f64422b-2448-4a97-9e1a-83f75e7fce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:13.879496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:53:14.430176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:14.431880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:14.496427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:53:14.510079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c86c9-a40f-4b71-82c2-45e1c39cceb3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:14.513646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:14.514252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:14.514525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:14.514792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:14.524315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:14.525765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef595ad-6e00-44e0-a576-e3bb90c983bc service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:14.534943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 767/1536] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:16.450554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:16.451155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:16.451155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:16.452469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 21:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:53:16.457584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b27a-f3eb-4ffa-bcec-0265cd5f83ab None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:16.458010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b27a-f3eb-4ffa-bcec-0265cd5f83ab None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:16.458160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b27a-f3eb-4ffa-bcec-0265cd5f83ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:16.478721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7820b27a-f3eb-4ffa-bcec-0265cd5f83ab None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:53:16.479950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7820b27a-f3eb-4ffa-bcec-0265cd5f83ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:16.487791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:53:17.040982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:53:17.042859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:53:17.056601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7897b-504d-48f1-9b0c-1f45354bf216 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:53:17.060134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:53:17.060734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:53:17.061015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:53:17.061288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:53:17.071715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:53:17.073151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42b00-37cd-4e61-b1a4-fa49a2a12f06 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:53:17.082820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 771/1539] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:53:26.472862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=11 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:53:27.078585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:40.084552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a310-fea5-4ada-8976-90cb723bf8d8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.085541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a310-fea5-4ada-8976-90cb723bf8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.085541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a310-fea5-4ada-8976-90cb723bf8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.108670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e0c2a310-fea5-4ada-8976-90cb723bf8d8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.110186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e0c2a310-fea5-4ada-8976-90cb723bf8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.121341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 769/1540] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:54:40.132355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.133230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.133394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.143511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4763fc-df58-454f-b3dc-654d1fc5e68a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.143768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4763fc-df58-454f-b3dc-654d1fc5e68a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.144109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4763fc-df58-454f-b3dc-654d1fc5e68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.162770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.164480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.177882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aa4763fc-df58-454f-b3dc-654d1fc5e68a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.178847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 772/1541] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:40.179988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa4763fc-df58-454f-b3dc-654d1fc5e68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.184173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.194862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 770/1542] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:54:40.198768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf24302-c67d-4009-8aff-8d315ed1ab38 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.203633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.203874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.204070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.204230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.204377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.204525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.205070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.205438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:40.205782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.206893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:54:40.206893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:54:40.206893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:54:40.207791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.234388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 773/1543] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:40.244363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.247805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.249010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.255850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162da21-fdf0-49f5-a4be-a2fddeddb823 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.266582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.266980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 771/1544] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:54:40.267329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.267583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.267956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.268608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.270554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804f7a1-6058-4171-8168-0a51a8f155d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.271799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.296756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063ce65-0794-4a21-b19c-eb5165b9caa1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.301365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.306181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.306517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.306785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.307387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:40.307685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.309299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.338725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 774/1546] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:40.338975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 772/1546] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:40.340186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.340479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.340740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.346853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.372125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac831-c3ed-4f07-8b9c-e2a4c6712d03 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.379042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.381236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.456819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 775/1547] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:40.457470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.493310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5d3b4-87f5-475b-b1cf-c36159d254d2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.514769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.515390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.515661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.515922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.517743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.519646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a9330-3aa5-4944-9770-afe823469cb8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.535391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.536015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.536707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.536995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.537815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.540008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501d2cf-7fd6-4ea5-b3e8-e1c4e930dc00 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.574525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 776/1548] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:40.576747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.583253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.583628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.607159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 773/1549] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:40.608468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.608766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.609033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.640135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.643513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.662625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.663897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.834726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 777/1550] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:40.835581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.844783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 774/1551] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:40.845817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.854199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7f130-c40c-4ed8-9548-7d8e640fa607 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.861838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5f90-4bce-487c-86ec-a1238ad51baa None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.864312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.865457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.865761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.866074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.868139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.869160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:54:40.869160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:54:40.869160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:54:40.873308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.875030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.875652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.875945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.876202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.876839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.878820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca24fef-97e0-4364-a65a-5b4938d943de admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.899148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 775/1552] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:40.904568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ec60-3d4b-4974-9b47-a9fb92fe88f9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.905097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ec60-3d4b-4974-9b47-a9fb92fe88f9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.905097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ec60-3d4b-4974-9b47-a9fb92fe88f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.925368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 778/1553] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:40.929803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:40.951428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11413-2af1-4d4e-bcc4-f3fa0b716cd1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:40.953757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2f82ec60-3d4b-4974-9b47-a9fb92fe88f9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:40.954979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2f82ec60-3d4b-4974-9b47-a9fb92fe88f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:40.963706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:40.964751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.965044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.965315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.965967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:40.966270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:40.967870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:40.976778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 776/1554] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:40.979150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:40.979443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:40.979716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:40.990526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:40.998754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 779/1555] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:40.999485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.022697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.024744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.031031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472044-990a-4e32-a355-92fa9999126b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.038374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 777/1556] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.039517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.040156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.041131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.041845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.042577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.043895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.047126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8887e-da51-4920-8ae3-beba4cb8d2ee admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.057727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1a1da-8656-4061-90be-77fb9e10bc1d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.064565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.065516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.066011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.066827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.067629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 780/1557] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:41.067943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.069927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538bd9-0ef5-44ce-84a2-1191da17bf3b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.070453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538bd9-0ef5-44ce-84a2-1191da17bf3b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.070878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538bd9-0ef5-44ce-84a2-1191da17bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.071012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.083714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 778/1558] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:41.084492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.092710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-91538bd9-0ef5-44ce-84a2-1191da17bf3b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.093938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-91538bd9-0ef5-44ce-84a2-1191da17bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.094308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8a2c-9f4e-48f0-82b8-0099cdaa0bc5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.099611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.100227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.100506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.100778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.101400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.102440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:54:41.102440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:54:41.102440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:54:41.102903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 781/1559] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.103654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.104135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8151dd-7333-4f4c-ad9e-c76b4c1e5eb1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.104553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8151dd-7333-4f4c-ad9e-c76b4c1e5eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.104734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8151dd-7333-4f4c-ad9e-c76b4c1e5eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.124251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5c8151dd-7333-4f4c-ad9e-c76b4c1e5eb1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.125298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5c8151dd-7333-4f4c-ad9e-c76b4c1e5eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.128666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:41.134129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 782/1560] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.135623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816bc4-cbe3-4413-9dc4-7687b5c2d449 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.135760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816bc4-cbe3-4413-9dc4-7687b5c2d449 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.135946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f816bc4-cbe3-4413-9dc4-7687b5c2d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.143688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 779/1561] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.144959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.160852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3f816bc4-cbe3-4413-9dc4-7687b5c2d449 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.162583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3f816bc4-cbe3-4413-9dc4-7687b5c2d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.207756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef687a5-ccf4-4774-a98d-230c8a7eb8d4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.212538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.213150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.213431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.213730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.214376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:41.214682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.216304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.222808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 783/1562] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.223838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.224649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.225272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.231412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 780/1563] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:41.232114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.242921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12168264-9203-40f2-81f3-a88182bfa1cd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.247515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.248185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.248457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.248723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] PATH_INFO: `/v3/projects/1c52341f519e47f59defbd74b623efac/users/a01cda302c5841cc937f219d7a06dc56/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.250670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.251888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.263064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 784/1564] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.265066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.266320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c52341f519e47f59defbd74b623efac, user_id=a01cda302c5841cc937f219d7a06dc56, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.267559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:54:41.267559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:54:41.267559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:54:41.270208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.274990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6291c-e5b0-4472-99f9-c306eb6f5937 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.280043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.280482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.280761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.281033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.281707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.283303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a58ef6-c120-4863-bd82-4056bc087cf5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.284231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 781/1565] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:41 2022] PUT /identity/v3/projects/1c52341f519e47f59defbd74b623efac/users/a01cda302c5841cc937f219d7a06dc56/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:41.285179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.294417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 785/1566] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:41.295640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f313d4-7520-4932-a1b3-19ea34efa42d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.295927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f313d4-7520-4932-a1b3-19ea34efa42d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.296202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f313d4-7520-4932-a1b3-19ea34efa42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.315770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-88f313d4-7520-4932-a1b3-19ea34efa42d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.316978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-88f313d4-7520-4932-a1b3-19ea34efa42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.352847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba821-a976-4550-a2f5-30114b5afa53 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.357565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.358205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.358487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.358760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.359400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:41.359704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.361308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.373241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 782/1567] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:41.374136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.380908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 786/1568] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.382566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.383030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.383312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.386603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d858866-9a7f-4748-ae59-b0dc76273be5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.391179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.391772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.392054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.392326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.392943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.394915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4ddc7-4dd5-4111-b57c-19baf7fa5a85 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.402353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.403547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.411733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 787/1569] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.412486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.417979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 783/1570] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.419699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eba82-b008-40f5-b6a8-cce3a36f2f35 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.419994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eba82-b008-40f5-b6a8-cce3a36f2f35 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.420272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eba82-b008-40f5-b6a8-cce3a36f2f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.428178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7a8d-779f-457e-a6b4-338dcbf02f09 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.433669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.434284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.434575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.434851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.435471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.437388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.440250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c24eba82-b008-40f5-b6a8-cce3a36f2f35 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.441469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c24eba82-b008-40f5-b6a8-cce3a36f2f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.452522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 784/1571] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.453998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.454280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.454551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.473334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 788/1572] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.474264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.475137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.476336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.507068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:41.511521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:41.538807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 785/1573] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.541808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e665f1c-4b4a-45b5-83ea-bb27d1b2674e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.543186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.547221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.548024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.548307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.549228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] PATH_INFO: `/v3/projects/3603d67da34f4bbfae5107398cfc235c/users/7608d6331ad042ef93aec56329b3e98f/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.552996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2165e-44f0-42c6-a10f-5c0e4cceddc6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.558043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.558654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.558931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.559201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.559821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.561795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.565266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3603d67da34f4bbfae5107398cfc235c, user_id=7608d6331ad042ef93aec56329b3e98f, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.568567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.594004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 789/1574] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:41 2022] PUT /identity/v3/projects/3603d67da34f4bbfae5107398cfc235c/users/7608d6331ad042ef93aec56329b3e98f/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:41.594004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.596472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 786/1575] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.597242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.661954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da08ab-eb0e-42a9-b254-55a915abd2ac admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.662141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145b3a-0746-4fb8-87a0-c012982af1a5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.666792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.667404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.667584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.667706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.667977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.668235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.668514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.668612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:41.668784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.668921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.669425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:41.669760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.670513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.671354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.683049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 790/1576] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:41.683698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.689754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 787/1577] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:41.691307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.696664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4b1d5-ae73-411f-bc4b-6b051c71c97c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.701769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d2b-537b-42aa-98fb-9bbec52d5f07 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.701935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.702411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.702691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.702965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.703606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.705228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.706645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.707557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.707557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.707773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.708416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:54:41.708762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.710389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.716537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 791/1578] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:41.717367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.730695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 788/1579] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:54:41.730695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.730981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203da4-5ebc-4466-974d-ecb8b496866f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.732373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.732978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.733259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.733533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.734212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.736171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d460-b928-4587-8f5e-693288ecaf17 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.758777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 792/1580] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/users => generated 284 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.760006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.760359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.760644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.783334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe0632-b7c2-4411-9bdc-27b003c0685c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.787519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.788126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.788359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] MFA Rules not processed for user `7608d6331ad042ef93aec56329b3e98f`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:41.788479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.788658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.789511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:41.797999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.799424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.809312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 789/1581] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:41.810126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.819220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82f88-0a2b-4936-bcfb-2f4c6c5869f1 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.824565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.825256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.825625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.825972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.826669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.828687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.852917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 790/1582] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.853763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.863299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad8db6-7158-47b5-aae5-7be835c9bb4d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.872814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 793/1583] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:41.873013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.873142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.873239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.873239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.873239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.873239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.873637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.884151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d5649-74a3-4c5d-8f0c-2099a6dbb2ff None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:41.889404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:41.890240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:41.890604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:41.890953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:41.891655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:41.893703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:41.899528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:41.906490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 791/1584] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.908234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.916228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:41.929630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 794/1585] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:41.930331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:41.991258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1cf5d-638d-4006-9b37-914b4fb6c741 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.003537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee280c5-4cb5-4c16-ba70-7f5d36e7c5e2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.012564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.013482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.014482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.014860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.016320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.018544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.055558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 795/1586] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:41 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.055558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.071949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.072665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.072981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.073306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.074034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.075679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.090071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 792/1587] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.090071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.103298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576ca62-438d-493d-9cbb-46ee4d5bff6d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.127299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95bb0-f3bd-4c3a-ae10-3f5c50f30de7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.133977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.135166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.135767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.135767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.136485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.139211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.155697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 796/1588] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.155697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.168350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0013d-eba5-4a54-87dd-71c0bda9a05d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.173673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.174533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.174666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.175157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.175298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.175483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.175589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.175741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.176181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.176360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.178033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.179207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.189712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 793/1589] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.190681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.202622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4d39-fb62-41cb-a3fa-88810f6c3222 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.208205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.209493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.209874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.210634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.211274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.213511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.217996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 797/1590] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.218982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.256031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 794/1591] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.257429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.270378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12389376-0299-4e12-a044-2de04aefba99 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.284829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:42.302856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:42.307205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2ffcf-7441-4850-8b34-47e87ecd35fd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.314897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.315678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.315962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.316240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.316924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.320126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.351645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 798/1592] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.352847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.354723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.355482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.355768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.356074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.356709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.358944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.365302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b9eb4-03b7-4970-8c76-17b4b417c42d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.370255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.371042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.371391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.371866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] PATH_INFO: `/v3/projects/efb811ce4fca47728e61489adbf97434/users/e280bab439a3446b993c721bc98c1c6e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.385639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 795/1593] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/users => generated 278 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.385639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.392221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb811ce4fca47728e61489adbf97434, user_id=e280bab439a3446b993c721bc98c1c6e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.395085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.397450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6da11-8f9b-4f4e-b110-c3f0db2d3d8b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.402959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.403737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.404157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.404486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.405308 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.407024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.415044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 799/1594] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:42 2022] PUT /identity/v3/projects/efb811ce4fca47728e61489adbf97434/users/e280bab439a3446b993c721bc98c1c6e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:42.415044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.419207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 796/1595] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.420191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.436051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f48b42-ee27-4eec-9640-3834d10711b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.497933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c9854-4c3e-468e-ae65-20725b78b1a5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.504242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.504462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.504950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.505078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.505254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.505941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.506045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.506243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.506701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.507035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.508421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.509436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.527178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 800/1596] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.528332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.540495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513d298-9484-48ae-a3cb-2fec991ffc2b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.545873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.546679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.546967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.547297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.548083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.551114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.554961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 797/1597] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:42.556437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.567140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 801/1598] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.568375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e2e54-8daa-43ca-926d-adb9a7e7134b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.568593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.574380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.574663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.574938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.575213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.575863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.577661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe66ea-59e9-4dc2-87b3-b1cdb442d72a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.579712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261285f-c159-443f-b66b-9a2bb3fc7d8e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.586365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.588031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.588248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.588479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.589630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.591156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.592129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 798/1599] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.593700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.593920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.594650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.610409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 802/1600] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.612208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.618771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:42.620022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:42.626461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e409-f401-4665-8cc5-9edcac10fc96 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.633006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.634044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.634567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.635186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] PATH_INFO: `/v3/projects/b8003e9995624ad3929a4ad2301c839a/users/0096583d70334cc5b185b51e0de9dd31/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.641724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 799/1601] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:42.641724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.660834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f2fe-7442-4083-adc4-3ccf8e5267a1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.667522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.668428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.668773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.669122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] PATH_INFO: `/v3/projects/6604f675c98a470db3568612140f7e8e/users/1bbaf0feaf294fc79f8f7d8dbcf58851/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.678260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8003e9995624ad3929a4ad2301c839a, user_id=0096583d70334cc5b185b51e0de9dd31, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.680992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.687826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6604f675c98a470db3568612140f7e8e, user_id=1bbaf0feaf294fc79f8f7d8dbcf58851, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.691717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.705616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 803/1602] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:42 2022] PUT /identity/v3/projects/b8003e9995624ad3929a4ad2301c839a/users/0096583d70334cc5b185b51e0de9dd31/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:42.713840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.722464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 800/1603] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:42 2022] PUT /identity/v3/projects/6604f675c98a470db3568612140f7e8e/users/1bbaf0feaf294fc79f8f7d8dbcf58851/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:42.723833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.788531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:42.806858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73203-9132-4288-a384-d9c30d1fa320 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.812924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.814407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.814691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.815014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.815662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.817287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.817493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b50c-93ab-4b62-a151-e9a00a3abf14 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.823387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.823995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.824280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.824565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.825323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.827151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.828781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:42.835102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 801/1604] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.836278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.844859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 804/1605] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.846263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.849983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2333a-7de0-48eb-b300-9288200f1ffd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.857394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.857732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0ee2-ec02-47c8-8803-b49bac6f2c63 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.858470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.858931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.859210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.859861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.861876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.865390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.866506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.867011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.867254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.867944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.869728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233817cb-d89e-4724-b5d2-e78df9c4cc23 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:42.878568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 802/1606] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.879653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:42.885011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 805/1607] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:42.887184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36da5c-045d-4af9-ae37-e41ef3265bb5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.887895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36da5c-045d-4af9-ae37-e41ef3265bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.888014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36da5c-045d-4af9-ae37-e41ef3265bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.922756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ac36da5c-045d-4af9-ae37-e41ef3265bb5 None None] MFA Rules not processed for user `0096583d70334cc5b185b51e0de9dd31`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:42.924003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ac36da5c-045d-4af9-ae37-e41ef3265bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:42.958915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71851-a301-45a0-9e96-900738eaaa5f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:42.964211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:42.964917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:42.965271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:42.965632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:42.981405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:42.983206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.001208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 806/1608] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:43.003636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.004033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.004451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.013859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 803/1609] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:43.016580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.027489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:43.028887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:43.043434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1df1e-ff62-4d0a-98e7-69523429324a service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.044048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 807/1610] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:43.045392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.055055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.055752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.056131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.057814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7988-91a3-41a1-980f-aa058188f25c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.060630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] PATH_INFO: `/v3/projects/e2e1b244604c4e698620022e4265b028/users/d461b953c9ee4901a1c144882d99da67/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.065435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.067258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.067828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.069688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] PATH_INFO: `/v3/projects/2ecfeccf26034651802ec8a1169f4697/users/9a61e9faee3949de8fb3830c57de6216/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.093617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecfeccf26034651802ec8a1169f4697, user_id=9a61e9faee3949de8fb3830c57de6216, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.096873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.105673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e1b244604c4e698620022e4265b028, user_id=d461b953c9ee4901a1c144882d99da67, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.109086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.131839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 808/1611] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:43 2022] PUT /identity/v3/projects/2ecfeccf26034651802ec8a1169f4697/users/9a61e9faee3949de8fb3830c57de6216/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:43.134037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.142712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 804/1612] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:43 2022] PUT /identity/v3/projects/e2e1b244604c4e698620022e4265b028/users/d461b953c9ee4901a1c144882d99da67/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:43.144944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.277654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661b52f-2740-4c42-880f-fe2b20d709d6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.287271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b4a34-6bc6-4ec3-af2a-e341f9e6d117 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.291753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.292614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.293089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.293456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] PATH_INFO: `/v3/projects/9894625c04a341e3a82584481410cd35/users/42072422001c4f1684f68baa25517a9e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.293628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.294181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.294548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.294905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] PATH_INFO: `/v3/projects/41f04f19ebae4309ac2de2e93339a07b/users/0f0720437be841f7b756ffc24546a31d/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.319487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f04f19ebae4309ac2de2e93339a07b, user_id=0f0720437be841f7b756ffc24546a31d, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.323344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.328306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9894625c04a341e3a82584481410cd35, user_id=42072422001c4f1684f68baa25517a9e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.331374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.357744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 809/1614] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:43 2022] PUT /identity/v3/projects/9894625c04a341e3a82584481410cd35/users/42072422001c4f1684f68baa25517a9e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:43.357744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.358444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 805/1614] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:43 2022] PUT /identity/v3/projects/41f04f19ebae4309ac2de2e93339a07b/users/0f0720437be841f7b756ffc24546a31d/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:43.362246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.365686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:43.448082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:43.452940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:43.501330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c68b2-f438-4d3c-b61c-42d80f3d9914 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.509899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c0a6-7f99-4b60-8216-15593971f767 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.523315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.524622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.524907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.525191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.604965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.606672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.606959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.607234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.642955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.644979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.674493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 810/1615] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:43.675510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:43.795802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.797343 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482a78e-b6db-4207-8b1d-17128174334b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.832615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 806/1616] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:43.834482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.834766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.835037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.853819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10338e17-8d33-46b8-b788-2377cdec86bc service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:43.859188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:43.859822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.860207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.860397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.861019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:43.862758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce53e3-e207-4939-8740-3fe797731403 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:43.879254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 811/1617] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:43.879254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307f96-a8a4-4c3e-9a32-40a2bfe2c1b3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.879254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307f96-a8a4-4c3e-9a32-40a2bfe2c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.879254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307f96-a8a4-4c3e-9a32-40a2bfe2c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.899222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] MFA Rules not processed for user `d461b953c9ee4901a1c144882d99da67`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:43.900470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:43.906407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c7307f96-a8a4-4c3e-9a32-40a2bfe2c1b3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:43.909666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c7307f96-a8a4-4c3e-9a32-40a2bfe2c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:43.937899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 812/1618] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:43.939129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:43.939430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:43.939690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:43.987448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] MFA Rules not processed for user `0f0720437be841f7b756ffc24546a31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:43.988924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:44.050349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 807/1619] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:44.051180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.072813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1102-34eb-4d61-8300-902db24fdf31 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.085413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.086257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.086723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.087124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] PATH_INFO: `/v3/projects/2ecfeccf26034651802ec8a1169f4697/users/9a61e9faee3949de8fb3830c57de6216/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.091330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecfeccf26034651802ec8a1169f4697, user_id=9a61e9faee3949de8fb3830c57de6216, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.095888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.099749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:44.104099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 813/1620] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:44.105873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.123843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78910ce2-05c9-42e9-9547-f5b8cd2768c8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.133872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 808/1621] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:44 2022] PUT /identity/v3/projects/2ecfeccf26034651802ec8a1169f4697/users/9a61e9faee3949de8fb3830c57de6216/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:44.134735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.202099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:44.254560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.255074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.258762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.259090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.333101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63b1ed-b068-4594-b5ad-e7311c50d23b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.347952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.349990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.350275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.350548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.351191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.357720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.424315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.425014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a10e4-d1d4-4ac6-9d18-df5938067287 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.442218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 809/1622] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:44 2022] POST /identity/v3/projects => generated 373 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:44.444136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.447606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 814/1623] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:44.449181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.449473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.449776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.514489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] MFA Rules not processed for user `9a61e9faee3949de8fb3830c57de6216`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:44.515881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:44.593456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96b2b-4579-4c81-9f36-e6df13d0a8b6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.607516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.608703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.608992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.609271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.613410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.622303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.663348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 815/1624] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:44.665334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.684310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d43a1a-a2a4-4b76-a80f-86924883795c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.699969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.701376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.701905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.702407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.703270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.705642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.711130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 810/1625] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:54:44 2022] POST /identity/v3/projects => generated 355 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:44.712042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.761803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 816/1626] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:44 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:44.761803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:44.789381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d26a-6dd3-4051-a416-2ee273dc32a2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.844733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:44.875747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99febadb-f5ce-4f36-bce1-97c422c6f960 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:44.890082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.890733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.891011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.891287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.937340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:44.938754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:44.939200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:44.939628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:44.940790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:44.945384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:44.979692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 817/1627] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:54:44 2022] POST /identity/v3/users => generated 284 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:54:44.980381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.014826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16427145-2c45-4e42-b6dd-352dd2109ac5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.015989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:45.037367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.038888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.039635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.040537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.040826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.041104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.043215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.053574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.061112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 811/1628] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:45.062074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.085562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f3cf-b412-497f-a89b-898ccc421fce service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.093701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 818/1629] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:45.094675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.096416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.097051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.097326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.097644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.098628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.100952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.118970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54737246-3553-4ad6-89e3-17af0c4fd973 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.127411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 812/1630] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:45.131139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.131746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.132189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.134729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.135217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] PATH_INFO: `/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b/users/bf5a03e34bd54dbca854b28020da55d2/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.153049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123a51-1675-4585-b0c2-0e506b3f39fa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.166364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.167049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.167901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.168575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] PATH_INFO: `/v3/projects/b4efd39529e546c486f4cf12698d987b/users/671e434c32ab4bd0937b5b25f4ac7f83/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.174674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d7ac02eb4d4bdfbaf4a1e94b1da51b, user_id=bf5a03e34bd54dbca854b28020da55d2, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.185917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.196940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4efd39529e546c486f4cf12698d987b, user_id=671e434c32ab4bd0937b5b25f4ac7f83, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.199673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.223096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 819/1631] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:45 2022] PUT /identity/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b/users/bf5a03e34bd54dbca854b28020da55d2/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:45.225926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.239115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 813/1632] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:45 2022] PUT /identity/v3/projects/b4efd39529e546c486f4cf12698d987b/users/671e434c32ab4bd0937b5b25f4ac7f83/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:45.240366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.313885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727189a-308d-4ca9-bc21-dcd3700bb9c4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.367688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56519b55-613c-4086-a5ce-4fcf2216239e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.378347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.378934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.379219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.379488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.380139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.385567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.392271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.392996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.393283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.394028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.394694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.396360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.411494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 814/1633] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:45.411723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 820/1634] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:54:45.412586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.415509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.428507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a88012-6c65-4d57-be24-550039f80917 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.434372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.435524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.435875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.436224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] PATH_INFO: `/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b/users/bf5a03e34bd54dbca854b28020da55d2/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.438114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951312d-4a8a-4342-82bf-2c91b7fc35c5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.438472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d7ac02eb4d4bdfbaf4a1e94b1da51b, user_id=bf5a03e34bd54dbca854b28020da55d2, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.444529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697c25-1e6b-4f8b-82e1-7c337c9808e4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.446675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.447375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.447657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.447931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] PATH_INFO: `/v3/projects/b4efd39529e546c486f4cf12698d987b/users/671e434c32ab4bd0937b5b25f4ac7f83/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.450626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4efd39529e546c486f4cf12698d987b, user_id=671e434c32ab4bd0937b5b25f4ac7f83, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.455957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a6e8a-b14d-4448-b352-64713f960fe8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.459346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 821/1635] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:45 2022] PUT /identity/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b/users/bf5a03e34bd54dbca854b28020da55d2/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:54:45.470654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.471264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.471739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.481110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 815/1636] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:54:45 2022] PUT /identity/v3/projects/b4efd39529e546c486f4cf12698d987b/users/671e434c32ab4bd0937b5b25f4ac7f83/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:54:45.492586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.494004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.494297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.496245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:45.532702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] MFA Rules not processed for user `bf5a03e34bd54dbca854b28020da55d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:45.535346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:45.574307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] MFA Rules not processed for user `671e434c32ab4bd0937b5b25f4ac7f83`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:45.575563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:45.695500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 822/1637] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:45.716252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:45.768364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 816/1638] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:45.858174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0ca7-6489-4325-9543-a8fa80b50a3c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:45.867097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:45.867896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:45.871737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:45.872148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:45.894539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:45.899085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:45.925525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 823/1639] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:45.989128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:46.015049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92840-d149-4aac-be14-5eea1e12fc54 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:46.032050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:46.034007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:46.034289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:46.035748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:46.061816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:46.063389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:46.091773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 817/1640] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:46.241366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:46.267383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33785e-9581-4071-8c77-371e963906ca service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:46.277156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:46.278115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:46.278652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:46.279081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:46.373218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:46.379103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:46.384040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad733bbc-268c-4c6f-8a26-6e7c648497f3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:46.399693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:46.400847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:46.402996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:46.404087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:46.417986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:46.419774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55269-b0cc-49c5-9d43-112437fde39f service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:46.445139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 824/1641] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:46.629543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:46.631458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f536c08-37c0-4115-8f5e-8aff2e66143a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:46.663590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 818/1642] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:54:46.689288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffe75-92ca-4d4a-a544-8e9d3147d236 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:46.690575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffe75-92ca-4d4a-a544-8e9d3147d236 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:46.691070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffe75-92ca-4d4a-a544-8e9d3147d236 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:46.704328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 825/1643] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 21:54:46 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:46.731602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b325-7db3-4517-88fa-3133a57408bf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:46.732159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b325-7db3-4517-88fa-3133a57408bf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:46.732608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b325-7db3-4517-88fa-3133a57408bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:46.864759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e262b325-7db3-4517-88fa-3133a57408bf None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:46.866014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e262b325-7db3-4517-88fa-3133a57408bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:47.050556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 819/1644] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 21:54:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:54:47.058104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d42c-5dd9-41bf-a651-1184d4173e6f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:47.058516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d42c-5dd9-41bf-a651-1184d4173e6f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:47.058645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7d42c-5dd9-41bf-a651-1184d4173e6f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:47.060296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 826/1645] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 21:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:47.066627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:47.066917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:47.067213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:47.118005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:47.119436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:47.172116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 820/1646] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:47.188829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7d42c-5dd9-41bf-a651-1184d4173e6f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:47.217234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7d42c-5dd9-41bf-a651-1184d4173e6f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:47.243607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:47.244943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:47.245545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:47.246151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:47.271444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:47.273839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600dc59-22d6-401c-a266-af3776f6681d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:47.291009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 827/1647] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:52.549491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:52.565805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9ea06-3f60-4872-af0f-b73ca91c14b9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:52.575006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:52.575655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:52.575937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:52.576559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:52.736071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:52.737904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9295c40-85f8-4b51-9b89-0fdcd8729f83 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:52.760788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 821/1648] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:52.966531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ff88-b5d6-452f-85fb-17654cf9a27a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:52.967525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ff88-b5d6-452f-85fb-17654cf9a27a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:52.968006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ff88-b5d6-452f-85fb-17654cf9a27a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:52.970139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 828/1649] 10.222.0.82 () {54 vars in 962 bytes} [Thu Aug 18 21:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:52.989162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:52.989789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:52.990394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:53.018810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:53.020120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:53.072209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cb572-7dd0-4ce4-8258-8f44b76a6ee8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:53.072565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cb572-7dd0-4ce4-8258-8f44b76a6ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:53.072886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cb572-7dd0-4ce4-8258-8f44b76a6ee8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:53.074532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 829/1650] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 21:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:53.080844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa124cb-58c6-440b-9a32-528f658a26f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:53.081128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa124cb-58c6-440b-9a32-528f658a26f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:53.081409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa124cb-58c6-440b-9a32-528f658a26f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:53.086836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 822/1651] 10.222.0.82 () {58 vars in 1059 bytes} [Thu Aug 18 21:54:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:53.162156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9aa124cb-58c6-440b-9a32-528f658a26f6 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:53.163769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9aa124cb-58c6-440b-9a32-528f658a26f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:53.218893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 830/1652] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:54:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:54:53.360065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:53.365575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:53.399084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95a47a-8a27-49df-acf7-b260fe222a59 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:53.421046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:53.424670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:53.425685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:53.425685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:53.483827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:53.485462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be51b5-964d-4bd2-8630-a63ab6085160 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:53.533320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 823/1653] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:54.626181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b5ff-2a30-49e7-932c-1263f0efaa57 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:54.626635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b5ff-2a30-49e7-932c-1263f0efaa57 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:54.626941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b5ff-2a30-49e7-932c-1263f0efaa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:54.675524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:54.702802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-baf4b5ff-2a30-49e7-932c-1263f0efaa57 None None] MFA Rules not processed for user `1bbaf0feaf294fc79f8f7d8dbcf58851`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:54.703598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-baf4b5ff-2a30-49e7-932c-1263f0efaa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:54.846616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 831/1654] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:54.860544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:54.863459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:54.863912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:54.918781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] MFA Rules not processed for user `1bbaf0feaf294fc79f8f7d8dbcf58851`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:54.920293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:54.923997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4073-792c-4dac-93ac-1c2d03c366aa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:54.924418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4073-792c-4dac-93ac-1c2d03c366aa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:54.924802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c4073-792c-4dac-93ac-1c2d03c366aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:54.944010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 824/1655] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:54.952726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:54.989147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076ea07-3fc4-48f2-bd28-83b2e562a909 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:54.992152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-681c4073-792c-4dac-93ac-1c2d03c366aa None None] MFA Rules not processed for user `a01cda302c5841cc937f219d7a06dc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:54.994459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-681c4073-792c-4dac-93ac-1c2d03c366aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:55.006328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:55.007073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:55.007537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:55.008882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:55.019835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:55.021859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:55.046715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 825/1656] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:55.168162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 832/1657] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:55.180210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:55.197119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378cbfa-4290-4dfb-afd2-cadee75087d1 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:55.219876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:55.220517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:55.220801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:55.221080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:55.258008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:55.260318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ef13-8f74-4e6c-88c9-8395fe085445 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:55.271729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 826/1658] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:55.827626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c105b0d-a19e-4b7d-9f54-8c118a306c1c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:55.828133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c105b0d-a19e-4b7d-9f54-8c118a306c1c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:55.828395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c105b0d-a19e-4b7d-9f54-8c118a306c1c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:55.830533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 833/1659] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 21:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:55.844058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc814e-7588-4462-a13e-0ca1448f93ee None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:55.844505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc814e-7588-4462-a13e-0ca1448f93ee None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:55.844888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc814e-7588-4462-a13e-0ca1448f93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:55.926074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c4cc814e-7588-4462-a13e-0ca1448f93ee None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:55.930561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c4cc814e-7588-4462-a13e-0ca1448f93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:56.066527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 827/1660] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:54:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:56.084871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c105b0d-a19e-4b7d-9f54-8c118a306c1c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:56.097261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c105b0d-a19e-4b7d-9f54-8c118a306c1c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:56.102632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:56.103534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:56.103962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:56.104541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:56.116128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:56.118169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:56.165483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 834/1661] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:56.884685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317eca-9380-49e7-9970-0a0083691933 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:56.885041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317eca-9380-49e7-9970-0a0083691933 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:56.885389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317eca-9380-49e7-9970-0a0083691933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:56.944444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-67317eca-9380-49e7-9970-0a0083691933 None None] MFA Rules not processed for user `e280bab439a3446b993c721bc98c1c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:56.955310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-67317eca-9380-49e7-9970-0a0083691933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:57.165468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 828/1662] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:57.173614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:57.194186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf438c0-0c1a-4045-a021-16381d30a6d9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:57.194475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf438c0-0c1a-4045-a021-16381d30a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:57.196043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf438c0-0c1a-4045-a021-16381d30a6d9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:57.197833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 829/1663] 10.222.0.82 () {54 vars in 961 bytes} [Thu Aug 18 21:54:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:54:57.204070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb4d3-c02c-4dd8-b195-500af2672f79 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:57.204358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb4d3-c02c-4dd8-b195-500af2672f79 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:57.204632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb4d3-c02c-4dd8-b195-500af2672f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:57.210915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c240-d48e-44c1-9f70-0bb56b046708 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:57.226723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:57.227364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:57.227646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:57.227963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:57.247568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1d2eb4d3-c02c-4dd8-b195-500af2672f79 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:57.249018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d2eb4d3-c02c-4dd8-b195-500af2672f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:57.258811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:57.264620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:57.292024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 835/1664] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:57.343442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:54:57.437172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 830/1665] 10.222.0.82 () {58 vars in 1058 bytes} [Thu Aug 18 21:54:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:57.449975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:57.468079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae94e-ecfd-471b-928f-c99d40722153 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:57.477585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:57.515643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:57.518982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:57.522828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:57.532103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:57.592181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:57.602489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28455e20-20cb-4f8c-809f-8dd24827b5ce service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:57.654880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 836/1666] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:59.061322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:59.061949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:59.061949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:59.156283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] MFA Rules not processed for user `42072422001c4f1684f68baa25517a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:59.159120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:59.355989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 831/1667] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:59.364205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f54cfa-81e4-47cb-a7a6-c211fa8bb8bc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:59.365244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f54cfa-81e4-47cb-a7a6-c211fa8bb8bc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:59.365381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f54cfa-81e4-47cb-a7a6-c211fa8bb8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:59.422343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-60f54cfa-81e4-47cb-a7a6-c211fa8bb8bc None None] MFA Rules not processed for user `42072422001c4f1684f68baa25517a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:59.423719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-60f54cfa-81e4-47cb-a7a6-c211fa8bb8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:54:59.446823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 837/1668] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:54:59.471277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:54:59.497100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d5479-f21d-46f5-9d63-72d902e2a49d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:54:59.514087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:54:59.514928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:59.515351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:59.515923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:59.537560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:54:59.540697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55c2c0-f693-4fd6-a40d-83aee5ef6970 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:54:59.561507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 832/1669] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:54:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:54:59.925220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b965-6c0e-4bd5-9075-e716563116ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:54:59.925833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b965-6c0e-4bd5-9075-e716563116ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:54:59.926292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b965-6c0e-4bd5-9075-e716563116ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:54:59.978717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0de6b965-6c0e-4bd5-9075-e716563116ce None None] MFA Rules not processed for user `a01cda302c5841cc937f219d7a06dc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:54:59.979937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0de6b965-6c0e-4bd5-9075-e716563116ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:00.023129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 838/1670] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:54:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:00.764409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb7f5-0e76-4bc6-af1a-70452f2fd966 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:00.764961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb7f5-0e76-4bc6-af1a-70452f2fd966 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:00.764961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb7f5-0e76-4bc6-af1a-70452f2fd966 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:00.766379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 833/1671] 10.222.0.82 () {54 vars in 912 bytes} [Thu Aug 18 21:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:55:00.773009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09397-e59e-4d8a-9696-7e4e01cb0571 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:00.773497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09397-e59e-4d8a-9696-7e4e01cb0571 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:00.773972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09397-e59e-4d8a-9696-7e4e01cb0571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:00.834340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fed09397-e59e-4d8a-9696-7e4e01cb0571 None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:00.837454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fed09397-e59e-4d8a-9696-7e4e01cb0571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:00.861176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 839/1672] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:55:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:00.929402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb7f5-0e76-4bc6-af1a-70452f2fd966 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:00.944328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb7f5-0e76-4bc6-af1a-70452f2fd966 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:00.954290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:00.955284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:00.955712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:00.956129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:00.973663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:00.980529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:01.010327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 834/1673] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:01.078832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e77f2-08c0-4a70-b8f1-0a74658bfedb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:01.079124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e77f2-08c0-4a70-b8f1-0a74658bfedb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:01.079422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e77f2-08c0-4a70-b8f1-0a74658bfedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:01.260935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:01.300228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ed6e77f2-08c0-4a70-b8f1-0a74658bfedb None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:01.308746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ed6e77f2-08c0-4a70-b8f1-0a74658bfedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:01.709424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 840/1674] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:55:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 632 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:02.031077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:02.032643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:02.436273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec1cbf-908c-4516-8a16-832ce9820874 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:02.455449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:02.464906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:02.467906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:02.468327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:02.515725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:02.517876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a611-4f69-4056-a64c-a513a1175a75 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:02.546787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 835/1675] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:55:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:55:03.440515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87787a-034f-471e-9339-12834dd9058c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:03.451632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:03.452154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:03.452242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:03.454081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87787a-034f-471e-9339-12834dd9058c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:03.454375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87787a-034f-471e-9339-12834dd9058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:03.548498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3d87787a-034f-471e-9339-12834dd9058c None None] MFA Rules not processed for user `e280bab439a3446b993c721bc98c1c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:03.554645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] MFA Rules not processed for user `42072422001c4f1684f68baa25517a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:03.555993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:03.558412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3d87787a-034f-471e-9339-12834dd9058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:03.619803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 841/1676] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:03.620137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 836/1677] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:06.844496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ae6d-78fa-4439-b86d-246039c94ad6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:06.845066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ae6d-78fa-4439-b86d-246039c94ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:06.845334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ae6d-78fa-4439-b86d-246039c94ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:06.898201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cdd3ae6d-78fa-4439-b86d-246039c94ad6 None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:06.902344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cdd3ae6d-78fa-4439-b86d-246039c94ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:06.920620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 842/1678] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 18 21:55:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:06.964295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:06.972621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:06.987337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be139f-5e15-4dfb-ab4b-31a2675c53db None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:06.997005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:06.999717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:07.000288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:07.002108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:07.035401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:07.039430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ad251-ae6e-471d-9440-b9d9be9d4a11 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:07.081919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 837/1679] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:09.886639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82121c7-b34c-4cca-a7d9-27259003da36 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:09.887948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82121c7-b34c-4cca-a7d9-27259003da36 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:09.888610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82121c7-b34c-4cca-a7d9-27259003da36 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:09.891506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 843/1680] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:55:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:55:09.904996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:09.906025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:09.906495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:09.980636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:09.981912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:09.997988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 838/1681] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:55:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:10.011643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82121c7-b34c-4cca-a7d9-27259003da36 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:10.031410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82121c7-b34c-4cca-a7d9-27259003da36 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:10.048220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:10.048943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:10.049286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:10.049568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:10.086582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:10.089428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef83b9-e506-4f95-b276-7ac66bdada28 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:10.109987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 844/1682] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:13.869382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:13.942973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df911f-3c45-457d-9005-3b985f99dca5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:13.948069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:13.948828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:13.949173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:13.949517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] PATH_INFO: `/v3/users/0096583d70334cc5b185b51e0de9dd31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:13.950778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0096583d70334cc5b185b51e0de9dd31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:13.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:55:13.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 18 21:55:13.951802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: Aug 18 21:55:13.953028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:14.090734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0096583d70334cc5b185b51e0de9dd31'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:14.093638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0096583d70334cc5b185b51e0de9dd31'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:14.095554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0096583d70334cc5b185b51e0de9dd31'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:14.164075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0096583d70334cc5b185b51e0de9dd31'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:14.194882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0096583d70334cc5b185b51e0de9dd31'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:14.241208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 839/1683] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:13 2022] DELETE /identity/v3/users/0096583d70334cc5b185b51e0de9dd31 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:14.242992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:14.243501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:14.246240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:14.296297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:14.301084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:14.544806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 845/1684] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:14.553349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:14.555288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:14.883799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db61ab-193e-4037-aa74-9bcc5e772184 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:14.911050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:14.912894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:14.918031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:14.919169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:14.952763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:14.963677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6a11d-74e4-407a-bab8-72e296b72b0d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:15.018291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 840/1685] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:16.068215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.070346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:16.083606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e764b4-fb4c-424c-b0cd-4f84b5a60016 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.088784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.089442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.089789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.090053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] PATH_INFO: `/v3/projects/b8003e9995624ad3929a4ad2301c839a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.091344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8003e9995624ad3929a4ad2301c839a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.092500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 21:55:16.092500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Aug 18 21:55:16.092500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: Aug 18 21:55:16.093821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.166640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8003e9995624ad3929a4ad2301c839a', 'request_id': 'req-26d6b283-ebcd-487b-9b9b-52186900485d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:16.178490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8003e9995624ad3929a4ad2301c839a', 'request_id': 'req-26d6b283-ebcd-487b-9b9b-52186900485d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:16.187281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8003e9995624ad3929a4ad2301c839a', 'request_id': 'req-26d6b283-ebcd-487b-9b9b-52186900485d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:16.207602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:16.219184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] The token cache is being invalidate because project b8003e9995624ad3929a4ad2301c839a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:16.220542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:16.222407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d6b283-ebcd-487b-9b9b-52186900485d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:16.230120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 846/1686] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:16 2022] DELETE /identity/v3/projects/b8003e9995624ad3929a4ad2301c839a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:16.230802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.231237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.231360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.248403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:16.256996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:16.259929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:16.355359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 841/1687] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:16.363398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.363687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.363962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.384786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:16.386266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:16.398542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 847/1688] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:16.403369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.418350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81d045-32f5-4ce3-a29b-b8e90241e920 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.424106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.424833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.425300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.433400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.436313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:16.437713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.454310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.468631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 842/1689] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:16.472780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.483224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9707a-da91-42a3-bc70-d26d84ff8271 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.489951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.489951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.490226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.490433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.491220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.504230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.561413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 848/1690] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/projects => generated 403 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:16.565999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.590717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdddc-5b05-4a1b-b1eb-f114584ee763 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.662185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.662930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.663289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.663667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.664402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.666741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.706149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 843/1691] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/users => generated 302 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:16.710608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.732043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138d8d-d163-4f82-85ad-f2f8d1b04f5c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.737763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.738402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.738683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.738980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.740159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.741854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.753554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 849/1692] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:16.759604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.770652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da89cb8-0e6d-4be6-a053-ec67923f34d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.776077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.776711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.776986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.777257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.777952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.780959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1a05-6295-4e2f-8159-c82de83ca106 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.798542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 844/1693] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:16.803097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:16.816407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acb4d-b788-419f-8508-6ed5dafc8da9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:16.820070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:16.820766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.821058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.821333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] PATH_INFO: `/v3/projects/4207e1f1d41b4000a1cfd07e6f24508f/users/5cfacf64d84f420a96f3d9446e1af867/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.838915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4207e1f1d41b4000a1cfd07e6f24508f, user_id=5cfacf64d84f420a96f3d9446e1af867, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:16.840997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:16.855577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 850/1694] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:16 2022] PUT /identity/v3/projects/4207e1f1d41b4000a1cfd07e6f24508f/users/5cfacf64d84f420a96f3d9446e1af867/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:16.868070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:16.868358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:16.868614 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:16.908747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] MFA Rules not processed for user `5cfacf64d84f420a96f3d9446e1af867`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:16.910006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:16.979201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 845/1695] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:16.983794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.047984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59fe3-2d0e-4675-aa50-ba4d0a5e0a28 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.053199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.053839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.054125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.054406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.055125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.057066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.084018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 851/1696] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:16 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:17.087979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.093516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:17.100761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a00473-dcc7-4905-906e-dd2c8843e596 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.155633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.156249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.156531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.156806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.157440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.159424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.186305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 846/1697] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:17 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:17.187674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.198015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18765733-0f02-48ad-97d0-fd3ae83d2f89 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.202810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.203417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.203698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.203983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.204620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.206248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.217407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 852/1698] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:17.224346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.233168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51792bbc-3f34-41b0-bfd7-c5e067ddce37 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.238185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.238859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.239207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.240204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] PATH_INFO: `/v3/projects/15147855cec9463e9363cf0e940f2c88/users/8ebee9cae0d641a789f9aedaa6d70773/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.255497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=15147855cec9463e9363cf0e940f2c88, user_id=8ebee9cae0d641a789f9aedaa6d70773, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.258243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.271750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 847/1699] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:17 2022] PUT /identity/v3/projects/15147855cec9463e9363cf0e940f2c88/users/8ebee9cae0d641a789f9aedaa6d70773/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:17.275784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.285716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a837ce8-ce4b-45af-bdb6-437af3a83b5c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.354834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.355378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.355665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.355944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.356623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.358289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88466db9-53c2-4588-ba5d-b0c30cf03942 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.368515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 853/1700] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:17.372850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.383254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f1b1-c5e9-4f28-bc39-3e0d5dabed4b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.387818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.388466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.388749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.389025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] PATH_INFO: `/v3/projects/15147855cec9463e9363cf0e940f2c88/users/8ebee9cae0d641a789f9aedaa6d70773/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.390978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] RBAC: Authorizing `identity:create_grant(project_id=15147855cec9463e9363cf0e940f2c88, user_id=8ebee9cae0d641a789f9aedaa6d70773, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.393680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.404298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 848/1701] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:17 2022] PUT /identity/v3/projects/15147855cec9463e9363cf0e940f2c88/users/8ebee9cae0d641a789f9aedaa6d70773/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:17.415069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.415371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.415963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.424125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:17.444365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] MFA Rules not processed for user `8ebee9cae0d641a789f9aedaa6d70773`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:17.445569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:17.551039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 854/1702] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:17 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:17.563882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.566125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:17.629809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda580-4d22-4a80-98cc-4a576824267c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:17.634460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:17.635098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:17.635347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:17.635622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:17.646217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:17.647576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48ba8c-c016-46ed-9d9e-c90025b11c96 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:17.658263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 849/1703] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:17 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:17.932126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:17.933786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:17.998338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d16f-b536-4f27-a4e1-360a3370f744 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:18.002764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:18.003389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.003667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.003938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] PATH_INFO: `/v3/users/0f0720437be841f7b756ffc24546a31d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.005078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0720437be841f7b756ffc24546a31d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:18.007071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:18.058973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0720437be841f7b756ffc24546a31d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.067650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0720437be841f7b756ffc24546a31d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.080386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0720437be841f7b756ffc24546a31d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.089143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0720437be841f7b756ffc24546a31d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.089885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0720437be841f7b756ffc24546a31d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.096403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 855/1704] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:17 2022] DELETE /identity/v3/users/0f0720437be841f7b756ffc24546a31d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:18.101461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.101817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.102079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.121470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:18.122695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.173472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:18.183788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 850/1705] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:18.193502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:18.195062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.205808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:18.253573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e78ac-c146-47e5-8ae2-fee5ef553e32 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:18.257237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:18.257875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.258157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.258430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.267298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:18.268715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df02a00-3fa0-4c68-b394-7b58bb0b4c9a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:18.278608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 856/1706] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:18.534474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:18.538123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.576864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8349e21-e93c-40c6-95a4-5d2de66542c6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:18.599101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:18.603877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.604386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.605091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] PATH_INFO: `/v3/users/5cfacf64d84f420a96f3d9446e1af867` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.609962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfacf64d84f420a96f3d9446e1af867)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:18.613124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:18.705021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfacf64d84f420a96f3d9446e1af867'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.705833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfacf64d84f420a96f3d9446e1af867'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.706458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfacf64d84f420a96f3d9446e1af867'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.723984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfacf64d84f420a96f3d9446e1af867'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.733661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfacf64d84f420a96f3d9446e1af867'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:18.750421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 851/1707] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:18 2022] DELETE /identity/v3/users/5cfacf64d84f420a96f3d9446e1af867 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:18.755648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.755942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.756215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.778327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:18.779570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.849191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 857/1708] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:18.898093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:18.910366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.939619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:18.941092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:18.955250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a2c7-391e-49e0-8241-b6d42bd9fadd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:18.960320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:18.961010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:18.961254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:18.961532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] PATH_INFO: `/v3/projects/41f04f19ebae4309ac2de2e93339a07b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:18.962730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f04f19ebae4309ac2de2e93339a07b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:18.975048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:19.067573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7284-f367-479e-93a0-32c623e52b68 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:19.089359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f04f19ebae4309ac2de2e93339a07b', 'request_id': 'req-496d30fc-e154-4447-898f-2b64e9be0024'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.108404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f04f19ebae4309ac2de2e93339a07b', 'request_id': 'req-496d30fc-e154-4447-898f-2b64e9be0024'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.117036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f04f19ebae4309ac2de2e93339a07b', 'request_id': 'req-496d30fc-e154-4447-898f-2b64e9be0024'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.152840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:19.153469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:19.153794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:19.154087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:19.159278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] The token cache is being invalidate because project 41f04f19ebae4309ac2de2e93339a07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:19.159587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.162434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.166859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 858/1709] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:18 2022] DELETE /identity/v3/projects/41f04f19ebae4309ac2de2e93339a07b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:19.170880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:19.174360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:19.206228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:19.235554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:19.237329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:19.259394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 852/1710] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:19.264673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:19.333791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d30fc-e154-4447-898f-2b64e9be0024 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:19.341382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:19.342573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:19.342702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:19.342941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] PATH_INFO: `/v3/users/671e434c32ab4bd0937b5b25f4ac7f83` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:19.344568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=671e434c32ab4bd0937b5b25f4ac7f83)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:19.348015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:19.418331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '671e434c32ab4bd0937b5b25f4ac7f83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.427389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '671e434c32ab4bd0937b5b25f4ac7f83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.440635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '671e434c32ab4bd0937b5b25f4ac7f83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.455384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '671e434c32ab4bd0937b5b25f4ac7f83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.456134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2ae807a3-6cc7-4ee1-a247-6ac2aad76a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '671e434c32ab4bd0937b5b25f4ac7f83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:19.463615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 859/1711] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:19 2022] DELETE /identity/v3/users/671e434c32ab4bd0937b5b25f4ac7f83 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:20.687768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:20.702101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969d0e-91d0-406a-888f-3c18b9e1e84a service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:20.784369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:20.785890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:20.786877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:20.787246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] PATH_INFO: `/v3/projects/b4efd39529e546c486f4cf12698d987b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:20.788915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4efd39529e546c486f4cf12698d987b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:20.815613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:20.919437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4efd39529e546c486f4cf12698d987b', 'request_id': 'req-92c86e58-e855-49d4-a012-4f9b9e142df1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:20.931469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4efd39529e546c486f4cf12698d987b', 'request_id': 'req-92c86e58-e855-49d4-a012-4f9b9e142df1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:20.955314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4efd39529e546c486f4cf12698d987b', 'request_id': 'req-92c86e58-e855-49d4-a012-4f9b9e142df1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:20.983454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] The token cache is being invalidate because project b4efd39529e546c486f4cf12698d987b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:20.985939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:20.988570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:20.998342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 853/1712] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:20 2022] DELETE /identity/v3/projects/b4efd39529e546c486f4cf12698d987b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:21.000462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140073a-9b15-4384-9925-12faaa7d1b07 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.001437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140073a-9b15-4384-9925-12faaa7d1b07 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.001925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140073a-9b15-4384-9925-12faaa7d1b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.053250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.054966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.113642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8140073a-9b15-4384-9925-12faaa7d1b07 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:21.118010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8140073a-9b15-4384-9925-12faaa7d1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.310914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c86e58-e855-49d4-a012-4f9b9e142df1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.312946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 860/1713] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:21.314681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109776db-efbb-48c7-b416-138df1d7fae9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.316057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109776db-efbb-48c7-b416-138df1d7fae9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.316659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109776db-efbb-48c7-b416-138df1d7fae9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.318703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 861/1714] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:55:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:55:21.323215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba737-2b32-46c6-88da-310e97a56131 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.323514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba737-2b32-46c6-88da-310e97a56131 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.323791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ba737-2b32-46c6-88da-310e97a56131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.324846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:21.325813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.326123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.326473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] PATH_INFO: `/v3/projects/4207e1f1d41b4000a1cfd07e6f24508f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.327770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4207e1f1d41b4000a1cfd07e6f24508f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:21.331184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:21.350590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:21.370420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-201ba737-2b32-46c6-88da-310e97a56131 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:21.373722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-201ba737-2b32-46c6-88da-310e97a56131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.385797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 862/1715] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:21.387850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.388294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.389111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.433410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:21.435829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.447123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4207e1f1d41b4000a1cfd07e6f24508f', 'request_id': 'req-a5297226-d04b-41dd-835e-7c4be19de73c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.466117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4207e1f1d41b4000a1cfd07e6f24508f', 'request_id': 'req-a5297226-d04b-41dd-835e-7c4be19de73c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.486440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4207e1f1d41b4000a1cfd07e6f24508f', 'request_id': 'req-a5297226-d04b-41dd-835e-7c4be19de73c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.514716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] The token cache is being invalidate because project 4207e1f1d41b4000a1cfd07e6f24508f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:21.515719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.518992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.522431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 854/1716] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:21 2022] DELETE /identity/v3/projects/4207e1f1d41b4000a1cfd07e6f24508f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:21.523473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.525966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:21.527491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.585525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 863/1717] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:55:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:21.588575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.591622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.684161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5297226-d04b-41dd-835e-7c4be19de73c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.699273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e281-92cb-4c83-8ff6-aeeadca610db None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.704968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:21.705725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.706013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.706291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] PATH_INFO: `/v3/users/8ebee9cae0d641a789f9aedaa6d70773` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.709504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebee9cae0d641a789f9aedaa6d70773)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:21.711808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:21.722019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:21.722669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.722953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.723227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.723998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:21.724305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:21.726016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:21.756556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 855/1718] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:21.772016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.799568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebee9cae0d641a789f9aedaa6d70773'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.807654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f826-d6ab-4a73-b9a3-a0b7e001c1cf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.809442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebee9cae0d641a789f9aedaa6d70773'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.815428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:21.816478 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.817031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.817966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.819364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:21.822075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:21.824389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebee9cae0d641a789f9aedaa6d70773'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.838257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebee9cae0d641a789f9aedaa6d70773'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.839981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebee9cae0d641a789f9aedaa6d70773'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:21.854011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 864/1719] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:21 2022] DELETE /identity/v3/users/8ebee9cae0d641a789f9aedaa6d70773 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:21.873225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 856/1720] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:21 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:21.874666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.883367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.890013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:21.891990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:21.980397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd0d12-0d3b-4774-9d61-0151d3858e49 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.983539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba8f03-500d-4a2e-9754-95a79ff66687 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:21.987918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:21.991652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:21.992065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:21.992398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:21.992620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:21.996270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.013727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:22.018767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.021847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.035972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 857/1721] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:22.093133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.093866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.094318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.094607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.095251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.097237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.121951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 865/1722] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:21 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:22.126172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.137738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20f1d1-a647-4cbf-a739-f79d2345999a service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.142361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.142981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.143259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.143534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.144181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.146014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.156866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 858/1723] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:22.161126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.174087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8413873-73d5-4b8d-9a66-cc7b8c179159 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.179804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.180433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.180781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.180974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.181714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.183311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44efa78-b0c8-4886-b51c-e84cf50aed55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.199360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 866/1724] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:22.203410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.214786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d0e1-6f36-4170-aad3-5ffc518a2ffb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.222983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.224620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.224984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.225334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] PATH_INFO: `/v3/projects/3a12bf44966e4b27a9a33e69e33fbd2a/users/ee9bbda2e8aa4c9ca9ad07ee4ca66a4c/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.246280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a12bf44966e4b27a9a33e69e33fbd2a, user_id=ee9bbda2e8aa4c9ca9ad07ee4ca66a4c, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.250110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.268154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 859/1725] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:22 2022] PUT /identity/v3/projects/3a12bf44966e4b27a9a33e69e33fbd2a/users/ee9bbda2e8aa4c9ca9ad07ee4ca66a4c/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:22.278864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.279160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.279618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.301169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.303916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.346576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] MFA Rules not processed for user `ee9bbda2e8aa4c9ca9ad07ee4ca66a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:22.349674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.385842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5ec5-85ec-4f0a-9f60-34833bc6a7d4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.393271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.394141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.394423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.395589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] PATH_INFO: `/v3/projects/15147855cec9463e9363cf0e940f2c88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.396889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15147855cec9463e9363cf0e940f2c88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.399106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.435507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 867/1726] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:22.439855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.450511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b491c-6f0b-4066-86a5-eaabea32dfc3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.458385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15147855cec9463e9363cf0e940f2c88', 'request_id': 'req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:22.463882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15147855cec9463e9363cf0e940f2c88', 'request_id': 'req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:22.471574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15147855cec9463e9363cf0e940f2c88', 'request_id': 'req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:22.483145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] The token cache is being invalidate because project 15147855cec9463e9363cf0e940f2c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:22.483432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:22.484067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9bf4bc8-01ef-49cd-8215-2aafab482c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:22.485860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 860/1727] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:22 2022] DELETE /identity/v3/projects/15147855cec9463e9363cf0e940f2c88 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:22.490869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd018cd-25bd-4900-aaba-0ca59c7a3947 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.491156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd018cd-25bd-4900-aaba-0ca59c7a3947 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.491432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd018cd-25bd-4900-aaba-0ca59c7a3947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.510844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3cd018cd-25bd-4900-aaba-0ca59c7a3947 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:22.510941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.511444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.511723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.511995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.512190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3cd018cd-25bd-4900-aaba-0ca59c7a3947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.512618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.514566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.520044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 861/1728] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:22.527842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.528128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.528404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.535650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:22.545766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 868/1729] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:22.547993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.549665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.549972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:22.551318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.592078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:22.615077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 862/1730] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:22.616587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59047160-c382-41b8-a425-52d9979e6485 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.619355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.621456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.622102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.622402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.622658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.623281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.625199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.628782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa3a92-3f24-4db0-a3e9-6f1e2bcce3bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.634091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.634787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.635167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.635484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.636238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:22.636649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.638560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.651441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 869/1731] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:22.652148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 863/1732] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:55:22.656590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.656812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.668611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833b373-3ddb-462b-9eee-d076d6e26450 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.669112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160925d-bf42-4b27-8d92-237334c6e407 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.674126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.674578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.674843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.675241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.675764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.675957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.676779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.677233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.677507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.677787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.678981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.681752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.688940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 870/1733] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:22.694479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.704621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e830-c8f0-4c36-a29a-6edc287edf8c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.711066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.711593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.711893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.712451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] PATH_INFO: `/v3/projects/067eb3db628f4856b676a3427b728fdc/users/8136b733484d4d49905222360ee00ff3/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.719352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 864/1734] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:22.723354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.729200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067eb3db628f4856b676a3427b728fdc, user_id=8136b733484d4d49905222360ee00ff3, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.731924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.744876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 871/1735] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:22 2022] PUT /identity/v3/projects/067eb3db628f4856b676a3427b728fdc/users/8136b733484d4d49905222360ee00ff3/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:22.748785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.788110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411341-9b4c-4e30-b8f6-6055e67d24c1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.812710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729ab39-14cd-44b6-998f-2c4108d297c1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.818182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.818862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.819324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.819782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.820638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.822269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.835728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 872/1736] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:22.836863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.846714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af7719-026b-4929-a4f1-44ee4fb4d9e7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.846989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.847668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.847885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.848159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.849332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.851281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.851909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.852558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.852839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.853115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] PATH_INFO: `/v3/projects/067eb3db628f4856b676a3427b728fdc/users/8136b733484d4d49905222360ee00ff3/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.855056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067eb3db628f4856b676a3427b728fdc, user_id=8136b733484d4d49905222360ee00ff3, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.857703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4f2d3-259d-401b-97c1-ae4013f1f575 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.869065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 873/1737] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:22 2022] PUT /identity/v3/projects/067eb3db628f4856b676a3427b728fdc/users/8136b733484d4d49905222360ee00ff3/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:22.878242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 865/1738] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:22.880309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.880601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.880882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.881986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.892451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c80e5-1357-4302-acb0-1f9a824736a4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.911075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] MFA Rules not processed for user `8136b733484d4d49905222360ee00ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:22.912804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:22.939725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:22.943292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:22.960570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.961356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.961664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.961944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.962719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.964348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:22.975798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 866/1739] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:22.979708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:22.990374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd4ab-f8a7-4a76-845a-f9bb9054cd1e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:22.995310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:22.995917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:22.996035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 874/1740] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:22.996192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:22.996479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:22.997123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:22.998750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.007058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.008689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:23.013556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 867/1741] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:23.014777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.024618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999777cf-c35d-467e-b8c7-33190a8c4cea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.029635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.029901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.030180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.030449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] PATH_INFO: `/v3/projects/d120806ca68e4e62a0f18c7f38cbaebb/users/aeb4f9a44cfc42f6b70c0ffa6bc444ce/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.047488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d120806ca68e4e62a0f18c7f38cbaebb, user_id=aeb4f9a44cfc42f6b70c0ffa6bc444ce, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.050536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0063f-dd1c-4732-803d-b4a0dfb7feb9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.070166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 868/1742] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:23 2022] PUT /identity/v3/projects/d120806ca68e4e62a0f18c7f38cbaebb/users/aeb4f9a44cfc42f6b70c0ffa6bc444ce/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:23.074839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a4f1a-13a2-4768-9c14-239a47e761f6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.082990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.083282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.083594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.124971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] MFA Rules not processed for user `aeb4f9a44cfc42f6b70c0ffa6bc444ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:23.126580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:23.143996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.144737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.145860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.146327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.201181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 869/1743] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:23.205375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.226258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.227711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.239708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 875/1744] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:23.276665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d918-f52a-4963-adaf-ac6b575b4d2b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.281192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.281833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.282115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.282396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.283051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.284958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.317473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 870/1745] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:23 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:23.320868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.321503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.332535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718a3a4-eeef-4a8d-9d86-0ca1bfdc5a38 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.348149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:23.380495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:23.395877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e9fdd-bcbc-4b70-9389-25b136c72ed9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.401394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.402054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.402429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.402689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.403255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.405385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.409135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.409872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.410228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.410593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.413271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:23.435357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 871/1746] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:23 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:23.436244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.454102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e24a7-5cd3-4aef-9dbe-1270a4908768 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.463142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.464541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.464831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.465111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.465799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.467733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.485607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 872/1747] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:23.487213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.504583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a745b35-2e0b-4bb6-bb59-b7940fc46e8d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.505580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.507740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.512134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.512618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.512945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.513833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] PATH_INFO: `/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9/users/150ee3bf2d804c74b25da5eac246ba53/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.519811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 876/1748] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:23.531003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e690c0cf1fdb4d7abbb7bc8fbee1e8b9, user_id=150ee3bf2d804c74b25da5eac246ba53, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.534981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.552984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 873/1749] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:23 2022] PUT /identity/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9/users/150ee3bf2d804c74b25da5eac246ba53/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:23.557667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.571126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50301abc-e990-4495-aa12-2852a791a3e0 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.638546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.639170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.639446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.639726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.640377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.642045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac9e01-9d17-4387-89a4-8df2c0e82c76 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.652614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 877/1750] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:23.656726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.668936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e625a6-1b6e-455d-a332-f4550baae2b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.673943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.674604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.674881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.675233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] PATH_INFO: `/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9/users/150ee3bf2d804c74b25da5eac246ba53/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.677767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e690c0cf1fdb4d7abbb7bc8fbee1e8b9, user_id=150ee3bf2d804c74b25da5eac246ba53, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.680498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.691294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 874/1751] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:23 2022] PUT /identity/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9/users/150ee3bf2d804c74b25da5eac246ba53/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:23.702315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.702605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.702885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.732314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] MFA Rules not processed for user `150ee3bf2d804c74b25da5eac246ba53`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:23.733570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:23.807139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 878/1752] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:23.818474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.874693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ed0d7-d0d6-408f-8ce8-a6dacffbb770 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.878346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.878951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.879231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.879503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:23.889746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:23.891175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:23.900560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 875/1753] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:23.976445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:23.982348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:23.986406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67ed86-50cf-4dba-b006-2286fa91b462 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:23.991074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:23.991677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:23.991961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:23.992234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:24.047586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:24.049028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab46e5-0abf-4fd3-b2ec-323ba4b45bff service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:24.065433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 879/1754] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:27.391726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:27.465031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1f945-1a4c-40e7-8957-34337e4609d6 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:27.470480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:27.471487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:27.471747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:27.472159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] PATH_INFO: `/v3/users/ee9bbda2e8aa4c9ca9ad07ee4ca66a4c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:27.474140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9bbda2e8aa4c9ca9ad07ee4ca66a4c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:27.477275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:27.483477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:27.560093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9bbda2e8aa4c9ca9ad07ee4ca66a4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:27.561179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9bbda2e8aa4c9ca9ad07ee4ca66a4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:27.562135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9bbda2e8aa4c9ca9ad07ee4ca66a4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:27.578141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9bbda2e8aa4c9ca9ad07ee4ca66a4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:27.587339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9bbda2e8aa4c9ca9ad07ee4ca66a4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:27.611903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 876/1755] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:27 2022] DELETE /identity/v3/users/ee9bbda2e8aa4c9ca9ad07ee4ca66a4c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:27.619489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:27.621465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:27.621856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:27.663288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:27.667826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:27.844197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 880/1756] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:27.848302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:27.851458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:27.967702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b970e5a-2cd2-46e2-ac71-e3283e29032a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:27.982802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:27.992193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:27.993087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:27.995727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.025050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.027009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2499d3-d9ce-4fe5-8cfa-407a0b98add1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.064713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 877/1757] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:28.136238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:28.137716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:28.152858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34a358-1cc6-4a34-9eb9-7d3071719f52 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:28.161963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:28.163228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.163228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.163415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] PATH_INFO: `/v3/users/7608d6331ad042ef93aec56329b3e98f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.168657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7608d6331ad042ef93aec56329b3e98f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.174460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.271579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7608d6331ad042ef93aec56329b3e98f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.280848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7608d6331ad042ef93aec56329b3e98f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.297496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7608d6331ad042ef93aec56329b3e98f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.308444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7608d6331ad042ef93aec56329b3e98f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.309198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7608d6331ad042ef93aec56329b3e98f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.317826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 881/1758] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:28 2022] DELETE /identity/v3/users/7608d6331ad042ef93aec56329b3e98f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:28.322302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.322734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.323458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.361837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:28.363387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:28.464004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 878/1759] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:28.479709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:28.481268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:28.505019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:28.561768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee1483-ff0f-4dc9-8951-76fe0071de0f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:28.566078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:28.566692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.566984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.567377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.578102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.579580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.591991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 882/1760] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:28.662580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:28.664662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:28.679978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a353fd9-c778-4eb2-ad75-be6e19a447ec None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:28.685385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:28.686264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.686619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.686822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] PATH_INFO: `/v3/projects/3a12bf44966e4b27a9a33e69e33fbd2a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.688325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a12bf44966e4b27a9a33e69e33fbd2a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.690391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.723315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:28.751813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a12bf44966e4b27a9a33e69e33fbd2a', 'request_id': 'req-afa4aa02-6d17-4147-bdfd-ba892afb280f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.758168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a12bf44966e4b27a9a33e69e33fbd2a', 'request_id': 'req-afa4aa02-6d17-4147-bdfd-ba892afb280f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.767660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a12bf44966e4b27a9a33e69e33fbd2a', 'request_id': 'req-afa4aa02-6d17-4147-bdfd-ba892afb280f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.784302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] The token cache is being invalidate because project 3a12bf44966e4b27a9a33e69e33fbd2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:28.784631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.785275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.788422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 879/1761] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:28 2022] DELETE /identity/v3/projects/3a12bf44966e4b27a9a33e69e33fbd2a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:28.792372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:28.802243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c582e-afc6-44c0-9157-da14078d46e4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:28.862768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:28.863405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.863691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.863957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] PATH_INFO: `/v3/users/8136b733484d4d49905222360ee00ff3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.865220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8136b733484d4d49905222360ee00ff3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.867201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.916730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:28.918202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:28.919537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8136b733484d4d49905222360ee00ff3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.927258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8136b733484d4d49905222360ee00ff3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.932099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4aa02-6d17-4147-bdfd-ba892afb280f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:28.937257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:28.937949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:28.938300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:28.938678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] PATH_INFO: `/v3/projects/3603d67da34f4bbfae5107398cfc235c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:28.939875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3603d67da34f4bbfae5107398cfc235c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:28.941948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:28.942786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8136b733484d4d49905222360ee00ff3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.953203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8136b733484d4d49905222360ee00ff3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.954048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8136b733484d4d49905222360ee00ff3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:28.962567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 883/1762] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:28 2022] DELETE /identity/v3/users/8136b733484d4d49905222360ee00ff3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:28.997828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3603d67da34f4bbfae5107398cfc235c', 'request_id': 'req-c3b12985-5d03-44d1-a9d2-439711f19faa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.007479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3603d67da34f4bbfae5107398cfc235c', 'request_id': 'req-c3b12985-5d03-44d1-a9d2-439711f19faa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.017528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3603d67da34f4bbfae5107398cfc235c', 'request_id': 'req-c3b12985-5d03-44d1-a9d2-439711f19faa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.035056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] The token cache is being invalidate because project 3603d67da34f4bbfae5107398cfc235c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:29.035471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.037041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.039336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 880/1763] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:28 2022] DELETE /identity/v3/projects/3603d67da34f4bbfae5107398cfc235c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:29.043481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.045132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:29.114847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:29.151729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60c93c-be6c-4a9d-9bd2-183cb391691a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.287370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.288027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.288526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.289248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] PATH_INFO: `/v3/users/9a61e9faee3949de8fb3830c57de6216` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.290181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a61e9faee3949de8fb3830c57de6216)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.292609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.321941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.323729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:29.340941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b12985-5d03-44d1-a9d2-439711f19faa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.352619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.353958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.354247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.354665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] PATH_INFO: `/v3/projects/067eb3db628f4856b676a3427b728fdc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.356361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=067eb3db628f4856b676a3427b728fdc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.360168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.377360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a61e9faee3949de8fb3830c57de6216'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.394280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a61e9faee3949de8fb3830c57de6216'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.427672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a61e9faee3949de8fb3830c57de6216'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.438922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067eb3db628f4856b676a3427b728fdc', 'request_id': 'req-b44b6cd8-501f-4f51-8921-75c08a10a875'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.445763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a61e9faee3949de8fb3830c57de6216'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.446577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b65c4c2c-456c-40f0-b439-56edce98dfb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a61e9faee3949de8fb3830c57de6216'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.448884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067eb3db628f4856b676a3427b728fdc', 'request_id': 'req-b44b6cd8-501f-4f51-8921-75c08a10a875'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.454848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 884/1764] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:29 2022] DELETE /identity/v3/users/9a61e9faee3949de8fb3830c57de6216 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:29.461536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067eb3db628f4856b676a3427b728fdc', 'request_id': 'req-b44b6cd8-501f-4f51-8921-75c08a10a875'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.476784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] The token cache is being invalidate because project 067eb3db628f4856b676a3427b728fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:29.477577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.477761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b44b6cd8-501f-4f51-8921-75c08a10a875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:29.479709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 881/1765] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:29 2022] DELETE /identity/v3/projects/067eb3db628f4856b676a3427b728fdc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:29.485023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.485301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.485585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.511071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:29.513628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:29.585438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 885/1766] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:29.593677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.594213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.594558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.615226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:29.616600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:29.626109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 882/1767] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:29.630699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.639648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:29.643624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072edcf-bb43-4458-8e4d-d2253511974a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.651403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.652223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.652690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.653287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.654218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:29.654851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.656466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.670262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 886/1768] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:29.683213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.710856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27c1ae-34d7-4409-8a0c-e6e8e5a26039 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.721230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.721897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.722199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.722456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.723087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.725064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.731291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=21,keystone:SELECT=76,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:29.756108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 883/1769] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:29 2022] POST /identity/v3/projects => generated 409 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:29.761142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.772512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36558d-6a0d-4d1e-ad4f-a52a489c3f5a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.838473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.839159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.839360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.839636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.840278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.842253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.866496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 887/1770] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:29 2022] POST /identity/v3/users => generated 305 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:29.870851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.882846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47803-5501-4a18-b4bd-a9f8a7789627 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.888178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.888796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.889070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.889340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.890020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.898611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.902881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.904967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:29.930301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77a57c-38d2-40f7-9032-6eb5bdb30eef admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.932050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 884/1771] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:29.933070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.936066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.936743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.937027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.937331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] PATH_INFO: `/v3/projects/2ecfeccf26034651802ec8a1169f4697` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.938550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecfeccf26034651802ec8a1169f4697)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.940523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.943671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791367a-681c-42bf-9606-68a96fc7cecc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.948384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:29.948991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:29.949275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:29.949579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:29.950244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:29.951844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:29.962492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 885/1772] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:29.966521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:29.976910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294ddea-8a8a-4c90-b7c4-566074bce904 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:29.993771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecfeccf26034651802ec8a1169f4697', 'request_id': 'req-6a220a6f-456f-41a9-8e9b-53f88bbb9774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:30.001196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecfeccf26034651802ec8a1169f4697', 'request_id': 'req-6a220a6f-456f-41a9-8e9b-53f88bbb9774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:30.008611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecfeccf26034651802ec8a1169f4697', 'request_id': 'req-6a220a6f-456f-41a9-8e9b-53f88bbb9774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:30.020366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] The token cache is being invalidate because project 2ecfeccf26034651802ec8a1169f4697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:30.021182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:30.021931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a220a6f-456f-41a9-8e9b-53f88bbb9774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:30.023455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 888/1773] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:29 2022] DELETE /identity/v3/projects/2ecfeccf26034651802ec8a1169f4697 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:30.028102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655d97-e40a-4e98-ade9-889be190abd1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.028388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655d97-e40a-4e98-ade9-889be190abd1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.028664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655d97-e40a-4e98-ade9-889be190abd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.035920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.036587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.036870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.037147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] PATH_INFO: `/v3/projects/a738d5c28b0f41c7a042a952fd28a797/users/21001b5ea58942b0bcd0c4411f895e81/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.047815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4a655d97-e40a-4e98-ade9-889be190abd1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:30.049027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4a655d97-e40a-4e98-ade9-889be190abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.053170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a738d5c28b0f41c7a042a952fd28a797, user_id=21001b5ea58942b0bcd0c4411f895e81, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.055885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77062-112a-4d4b-8e63-a64ddd50ef69 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.058071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 889/1774] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:30.067826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.068129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.068408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.071804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 886/1775] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:29 2022] PUT /identity/v3/projects/a738d5c28b0f41c7a042a952fd28a797/users/21001b5ea58942b0bcd0c4411f895e81/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:30.081926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.082212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.082489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.090460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:30.091681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.111319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] MFA Rules not processed for user `21001b5ea58942b0bcd0c4411f895e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:30.112530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.143482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:30.158414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 890/1776] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:30.159276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.169032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042c1bd-a78d-410f-9324-77233d59b8b5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.173684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.174279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.174581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.174859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.175492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:30.175793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.177500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.180205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 887/1777] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:30.184557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.185903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.190820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 891/1778] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:30.195511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.199410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108afe7f-d1d1-4241-95ff-db9138afe1de None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.204459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.205069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.205162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e97aa-1616-403a-93b8-72a35123871e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.205351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.205643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.206269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.208701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.210533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.211150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.211456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.211725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.212335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.214802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.238536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 888/1779] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:30.242332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.249883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 892/1780] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:30.250852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.263547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:30.310338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b08a1-bc53-41a8-8a12-2a87c7a5c02c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.321533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112248b-2355-46c6-9d7e-63172b3c9add admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.326736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.327385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.327709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.327993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.328687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.330843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.361626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 893/1781] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:30.362865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.373924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654dbea-d13d-4d4f-9a27-df6c6c2a9f55 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.378219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.378751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.379248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.379354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.379686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.380468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.380746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.381144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.381356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.382229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.382781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.384808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.393645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 894/1782] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:30.397970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.409078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f96aa-ca85-49cb-bf42-b7b25c2dcfea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.413155 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 889/1783] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:30.417284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.417426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.417948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.418230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.418507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.419151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.420743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.427774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29adec9-d9b5-4b5d-8126-1c7e69384313 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.432238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.432857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.433138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.433410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.434739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.436332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.438933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 895/1784] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:30.443498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.456184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79943770-815e-4c83-9d4c-1eebb55f0cf2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.461787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 890/1785] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:30.464462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.467202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.467877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.468162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.468437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] PATH_INFO: `/v3/projects/8cb944031a954726851c308cf2d4601e/users/6db00070278b49c1970ea6dd1baea58e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.481325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962775-1360-4733-9d5d-bdd9bbe542e1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.488421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.489903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.490490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.490767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] PATH_INFO: `/v3/projects/63e11634726f45749af72b1e592de0dd/users/aca9c3d145114468849f521772a7035c/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.496138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb944031a954726851c308cf2d4601e, user_id=6db00070278b49c1970ea6dd1baea58e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.498636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed1a96-2df0-4ffb-9692-a2349c4d2a66 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.509290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e11634726f45749af72b1e592de0dd, user_id=aca9c3d145114468849f521772a7035c, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.512408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.515323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 896/1786] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:30 2022] PUT /identity/v3/projects/8cb944031a954726851c308cf2d4601e/users/6db00070278b49c1970ea6dd1baea58e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:30.530347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.530631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.531095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.534043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 891/1787] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:30 2022] PUT /identity/v3/projects/63e11634726f45749af72b1e592de0dd/users/aca9c3d145114468849f521772a7035c/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:30.538262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.559677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] MFA Rules not processed for user `6db00070278b49c1970ea6dd1baea58e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:30.560904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.610364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b58b1-b31e-4b88-8567-3a15121ec528 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.616351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.616960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.617236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.617505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.618192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.619692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:30.619800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.633728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 892/1788] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:30.635275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.635781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 897/1789] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:30.640089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.647498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fed89b-89a7-4a78-889f-60676a631cb5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.653513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b8704-d36b-432a-bef1-74f758f541d5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.656166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.656879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.657120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.657400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] PATH_INFO: `/v3/projects/63e11634726f45749af72b1e592de0dd/users/aca9c3d145114468849f521772a7035c/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.658915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.659672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.660024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.660365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.660582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e11634726f45749af72b1e592de0dd, user_id=aca9c3d145114468849f521772a7035c, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.661178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.663211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.665456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457052b0-af5e-44c7-85f9-245bfbc72714 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.679913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:30.685247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 893/1790] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:30 2022] PUT /identity/v3/projects/63e11634726f45749af72b1e592de0dd/users/aca9c3d145114468849f521772a7035c/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:30.697589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.697894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.698169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.705563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 898/1791] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:30.710208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.748162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] MFA Rules not processed for user `aca9c3d145114468849f521772a7035c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:30.749522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.809533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def75796-2ecf-4189-a36f-41ecf2df3fa3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.814638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.817799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.817799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.817799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.817799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.819092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.838176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 894/1792] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:30.849480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.851394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:30.853282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 899/1793] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:30 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:30.857149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.873031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317923-0709-4e05-a452-c16baaf8e215 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.878935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.879663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.880544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.880815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.881638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.883272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.895830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 900/1794] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:30.899822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:30.913349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a48920-5ef0-44e5-b999-b541416b97da admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.927913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f124a-102c-449e-83f6-3c5c768cc7b3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:30.929239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.930125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.930351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.930629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] PATH_INFO: `/v3/projects/6ead0c2488d54697b9ebd60af368d340/users/70feeb0d467a4a9facaceaf1bc7eb3e8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.933080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:30.933956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:30.934302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:30.934583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:30.952218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.953918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.954871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ead0c2488d54697b9ebd60af368d340, user_id=70feeb0d467a4a9facaceaf1bc7eb3e8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:30.958606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:30.970791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 895/1795] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:30.978299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 901/1796] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:30 2022] PUT /identity/v3/projects/6ead0c2488d54697b9ebd60af368d340/users/70feeb0d467a4a9facaceaf1bc7eb3e8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:30.982975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:31.007554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44f1cc-c65c-4051-9911-b1aa4915e85f service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:31.163182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:31.168515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.168847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.169763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.175722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:31.183150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953d21-4262-45fa-87f5-78f2e1c350eb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:31.221778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 896/1797] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:31.226170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:31.240450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cd8fe-09dc-4bbb-ba8b-fc91c602e0af admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:31.255384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:31.256091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.256385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.256669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] PATH_INFO: `/v3/projects/6ead0c2488d54697b9ebd60af368d340/users/70feeb0d467a4a9facaceaf1bc7eb3e8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.258979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ead0c2488d54697b9ebd60af368d340, user_id=70feeb0d467a4a9facaceaf1bc7eb3e8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:31.261973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:31.306227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 902/1798] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:31 2022] PUT /identity/v3/projects/6ead0c2488d54697b9ebd60af368d340/users/70feeb0d467a4a9facaceaf1bc7eb3e8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:31.310105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.311296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.311988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.363355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] MFA Rules not processed for user `70feeb0d467a4a9facaceaf1bc7eb3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:31.365877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:31.400506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:31.442488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:31.563297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 897/1799] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:31.564118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfab19-fff3-48ab-bb56-71aa7fb4b964 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:31.569305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:31.570137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.570825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.571102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.609819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:31.611342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:31.620305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:31.626385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad410-5bb3-4fc7-958f-2ac45d3b0816 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:31.630830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:31.631523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.631828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.632104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.643251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:31.644920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:31.662826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 898/1800] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:31.663987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:31.666504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc80ec-8651-43fe-bc50-faf5e661c265 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:31.678641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 903/1801] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:31.809517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:31.823264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed29a-77db-4b1d-aac2-df6012d01c05 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:31.829258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:31.829977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:31.830255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:31.845622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:31.963453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:31.974345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2dbde-f65d-4785-81ee-fcbf06a966f1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:31.993765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 899/1802] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:33.485478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:33.486018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:33.486123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:33.487779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 904/1803] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 21:55:33.500588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:33.501011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:33.501083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:33.524431 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:33.525725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:33.616422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 900/1804] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:55:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:33.649021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:33.650816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:33.681021 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69465-296c-402b-80a0-b12c0cc67533 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:33.688580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:33.689254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:33.689642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:33.689895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:33.701837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:33.703526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efac48c-d110-46ec-a320-d5e861be3aa1 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:33.716999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 905/1805] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:35.902398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:35.933695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:35.966176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5d6f-a522-478d-a697-2fa74e1cae8c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:35.970547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:35.971170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:35.971445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:35.971716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] PATH_INFO: `/v3/users/6db00070278b49c1970ea6dd1baea58e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:35.972814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db00070278b49c1970ea6dd1baea58e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:35.974800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:36.018618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db00070278b49c1970ea6dd1baea58e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.019268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db00070278b49c1970ea6dd1baea58e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.019879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db00070278b49c1970ea6dd1baea58e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.032352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db00070278b49c1970ea6dd1baea58e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.039248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db00070278b49c1970ea6dd1baea58e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.052750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 901/1806] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:35 2022] DELETE /identity/v3/users/6db00070278b49c1970ea6dd1baea58e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:36.057344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:36.057841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:36.057929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:36.077033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:36.078309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:36.135600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 906/1807] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:36.145221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:36.146650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:36.204760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e243ec-5ac7-4e95-896a-57cc8c4e34d2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:36.209062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:36.209557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:36.209964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:36.210288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:36.220674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:36.222848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:36.232586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 902/1808] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:36.504691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:36.506185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:36.518120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51c30f-77c7-40e9-a7d2-647167a25aaa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:36.523692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:36.524409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:36.525013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:36.525013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] PATH_INFO: `/v3/projects/8cb944031a954726851c308cf2d4601e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:36.526553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb944031a954726851c308cf2d4601e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:36.528624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:36.592899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb944031a954726851c308cf2d4601e', 'request_id': 'req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.606709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb944031a954726851c308cf2d4601e', 'request_id': 'req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.630416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb944031a954726851c308cf2d4601e', 'request_id': 'req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.649316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] The token cache is being invalidate because project 8cb944031a954726851c308cf2d4601e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:36.649627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.650468 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.652741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 907/1809] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:36 2022] DELETE /identity/v3/projects/8cb944031a954726851c308cf2d4601e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:36.657140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:36.669387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e838b0-1546-45cc-ae5b-01aa9f7cdd24 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:36.746461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:36.768085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:36.768227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:36.768512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:36.768788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] PATH_INFO: `/v3/users/70feeb0d467a4a9facaceaf1bc7eb3e8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:36.769996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=70feeb0d467a4a9facaceaf1bc7eb3e8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:36.778739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:36.895197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70feeb0d467a4a9facaceaf1bc7eb3e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.897440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70feeb0d467a4a9facaceaf1bc7eb3e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.898118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70feeb0d467a4a9facaceaf1bc7eb3e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.928606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70feeb0d467a4a9facaceaf1bc7eb3e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.951876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b547421e-4c91-438e-842d-6732c072faeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70feeb0d467a4a9facaceaf1bc7eb3e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:36.983637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 903/1810] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:36 2022] DELETE /identity/v3/users/70feeb0d467a4a9facaceaf1bc7eb3e8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:37.449876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:37.452850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:37.503973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400d78-b1f7-4c1f-96c6-0a43e0abf213 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:37.555662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:37.596518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:37.597284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.597961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.599019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] PATH_INFO: `/v3/projects/6ead0c2488d54697b9ebd60af368d340` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.600608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ead0c2488d54697b9ebd60af368d340)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:37.604704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:37.683237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ead0c2488d54697b9ebd60af368d340', 'request_id': 'req-84f7ee27-b967-48f5-9dba-ed9ea2a77239'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:37.703521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ead0c2488d54697b9ebd60af368d340', 'request_id': 'req-84f7ee27-b967-48f5-9dba-ed9ea2a77239'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:37.721493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ead0c2488d54697b9ebd60af368d340', 'request_id': 'req-84f7ee27-b967-48f5-9dba-ed9ea2a77239'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:37.736513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] The token cache is being invalidate because project 6ead0c2488d54697b9ebd60af368d340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:37.736907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:37.737683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84f7ee27-b967-48f5-9dba-ed9ea2a77239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:37.739575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 908/1811] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:37 2022] DELETE /identity/v3/projects/6ead0c2488d54697b9ebd60af368d340 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:37.746001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.746452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.746933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.768970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:37.770768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:37.841036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 904/1812] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:37.854291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.855126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.855126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.868990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:37.870506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:37.877075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:37.878365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:37.891559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efac45-ed65-4a10-90bd-2309211e9513 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:37.896094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 909/1813] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:37.898019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:37.907559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:37.909096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.909922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.910676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] PATH_INFO: `/v3/users/aeb4f9a44cfc42f6b70c0ffa6bc444ce` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.913499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb4f9a44cfc42f6b70c0ffa6bc444ce)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:37.917056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d3c09-7e54-42cc-91ca-39cb3f9358f8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:37.918105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:37.925906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:37.927065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.927658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.928233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.929515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:37.930188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:37.933892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:37.951772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 910/1814] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:55:37.956163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:37.967962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a2b4-35ca-4693-ad2d-84973aa051a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:37.976622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:37.977455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:37.977908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:37.978321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:37.979116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:37.981273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.006072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb4f9a44cfc42f6b70c0ffa6bc444ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.007485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb4f9a44cfc42f6b70c0ffa6bc444ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.008287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb4f9a44cfc42f6b70c0ffa6bc444ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.021153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:38.022806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb4f9a44cfc42f6b70c0ffa6bc444ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.033204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9736303-726e-4d37-9a13-8869480462d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb4f9a44cfc42f6b70c0ffa6bc444ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.039505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 911/1815] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:37 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:38.043965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.045965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.062210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 905/1816] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:37 2022] DELETE /identity/v3/users/aeb4f9a44cfc42f6b70c0ffa6bc444ce => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:38.066959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.067775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.068053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.101061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:38.103769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.132382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6543c-f1eb-4d88-8a16-c8b01bc85a64 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.170430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:38.185149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 906/1817] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:38.195073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.197824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.204273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.204904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.205190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.205466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.206141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.208243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.235743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 912/1818] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:38 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:38.240238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.254946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36fdf-e224-4113-acca-b2b3ad77fc10 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.274662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.275310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.275613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.275880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.276542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.277077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19330f6-a49d-47bb-b006-9c2bc1b54cbc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.278211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.281542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.282962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.283380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.283660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.294065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 913/1819] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:38.296711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.299205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.300666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.308067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7589453-9582-48be-8ebe-6234b340e09a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.311837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 907/1820] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:38.314609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.315227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.315512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.315787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.316483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.318151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a55d5-ae9c-4135-9473-7bffda30a240 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.330061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 914/1821] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:38.334863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.345304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455e9dc-9fec-4e9a-b2e2-09aab66249ba service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.350870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.351601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.351952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.352353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] PATH_INFO: `/v3/projects/f8dfd1ab64c340fe94d34f6f745717a8/users/2027d81390a14f5db200e21058a04103/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.368394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dfd1ab64c340fe94d34f6f745717a8, user_id=2027d81390a14f5db200e21058a04103, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.371140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.393121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 908/1822] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:38 2022] PUT /identity/v3/projects/f8dfd1ab64c340fe94d34f6f745717a8/users/2027d81390a14f5db200e21058a04103/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:38.395059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.396023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.396299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.426628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] MFA Rules not processed for user `2027d81390a14f5db200e21058a04103`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:38.427847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.498217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 915/1823] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:38.504394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.600807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0832834-1903-4081-96a6-63e5e302cc9f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.606216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.606777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.607061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.607394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.608031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.610097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.617123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:38.663228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 909/1824] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:38 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:38.664619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.675784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb0b2-776c-4b1c-956b-d263d32b53f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.711176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.713457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.771843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.772471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.772754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.773029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.773694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.775689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.785495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a1eb2-f7a9-4760-8d0d-0315edab6689 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.791829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.792137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.792537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.792803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] PATH_INFO: `/v3/projects/d120806ca68e4e62a0f18c7f38cbaebb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.793985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d120806ca68e4e62a0f18c7f38cbaebb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.795890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.797893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 916/1825] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:38 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:38.801875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.811743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1c15-19cb-4b66-aa7f-10ca2cf8a28e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.816484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.817622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.817926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.818201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.818839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.820567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.838633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 917/1826] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:38.839832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.849464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44505-2f6e-4604-b3be-b643c050d58c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.850398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd120806ca68e4e62a0f18c7f38cbaebb', 'request_id': 'req-76b43054-6e04-4e4c-9407-ce34681779ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.856248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd120806ca68e4e62a0f18c7f38cbaebb', 'request_id': 'req-76b43054-6e04-4e4c-9407-ce34681779ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.864330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:38.865362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd120806ca68e4e62a0f18c7f38cbaebb', 'request_id': 'req-76b43054-6e04-4e4c-9407-ce34681779ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.877178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] The token cache is being invalidate because project d120806ca68e4e62a0f18c7f38cbaebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:38.877474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.878204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:38.880823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 910/1827] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:38 2022] DELETE /identity/v3/projects/d120806ca68e4e62a0f18c7f38cbaebb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:38.884794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.886662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.918606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:38.919278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:38.919559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:38.919839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] PATH_INFO: `/v3/projects/ca496a297a114f2da76e37b63e7500af/users/a87adc270cd245d78bf8544ab8a7dfc2/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:38.935188 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca496a297a114f2da76e37b63e7500af, user_id=a87adc270cd245d78bf8544ab8a7dfc2, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:38.937888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:38.951073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 918/1828] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:38 2022] PUT /identity/v3/projects/ca496a297a114f2da76e37b63e7500af/users/a87adc270cd245d78bf8544ab8a7dfc2/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:38.954349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43054-6e04-4e4c-9407-ce34681779ae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:38.955306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:38.956941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:38.975859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:39.017969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.018648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.018846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.019124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] PATH_INFO: `/v3/users/150ee3bf2d804c74b25da5eac246ba53` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.020302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=150ee3bf2d804c74b25da5eac246ba53)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.022845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.028536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553f8eb-6186-40a2-8148-27dab70f704e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:39.033415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.034067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.034345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.034618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.035255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.037332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.047542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 919/1829] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:39.052549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:39.065401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ddcbe-e4ab-4e52-be6f-ac8323c2f4b9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:39.070744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.072161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.072299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.072943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] PATH_INFO: `/v3/projects/ca496a297a114f2da76e37b63e7500af/users/a87adc270cd245d78bf8544ab8a7dfc2/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.074130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '150ee3bf2d804c74b25da5eac246ba53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.074750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '150ee3bf2d804c74b25da5eac246ba53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.075100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca496a297a114f2da76e37b63e7500af, user_id=a87adc270cd245d78bf8544ab8a7dfc2, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.075367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '150ee3bf2d804c74b25da5eac246ba53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.078228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cef47-f9d1-496d-9966-726a3c044f20 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.089559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '150ee3bf2d804c74b25da5eac246ba53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.092002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 920/1830] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:39 2022] PUT /identity/v3/projects/ca496a297a114f2da76e37b63e7500af/users/a87adc270cd245d78bf8544ab8a7dfc2/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:39.097140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '150ee3bf2d804c74b25da5eac246ba53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.104199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.104581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.104800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.115973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 911/1831] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:38 2022] DELETE /identity/v3/users/150ee3bf2d804c74b25da5eac246ba53 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:39.136589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] MFA Rules not processed for user `a87adc270cd245d78bf8544ab8a7dfc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:39.138075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:39.230383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 921/1832] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:39.241961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:39.243592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:39.339090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dc9ba-cec4-4ea2-9607-4c98ca249f90 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:39.350487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.351123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.351421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.351697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.361511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.363032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.373485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 912/1833] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:39.494640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:39.500461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:39.508224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eab744-e859-40b8-8abf-6d2d2391a163 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:39.517171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.518370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.519163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.519705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.522848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:39.600103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.602917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767533e9-de0a-4248-b842-27698174bc43 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.618471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 922/1834] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:39.620035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:39.622781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:39.705199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:39.741054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d99c-c2fb-4397-96cc-7e2a7066a198 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:39.753566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:39.754864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.755086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.755361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] PATH_INFO: `/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.762621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e690c0cf1fdb4d7abbb7bc8fbee1e8b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:39.765742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:39.836367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e690c0cf1fdb4d7abbb7bc8fbee1e8b9', 'request_id': 'req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.844160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e690c0cf1fdb4d7abbb7bc8fbee1e8b9', 'request_id': 'req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.857767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e690c0cf1fdb4d7abbb7bc8fbee1e8b9', 'request_id': 'req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.870880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] The token cache is being invalidate because project e690c0cf1fdb4d7abbb7bc8fbee1e8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:39.871351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.872180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c0d276e5-c5f3-4fee-8cb6-b919e172bba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:39.874776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 913/1835] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:39 2022] DELETE /identity/v3/projects/e690c0cf1fdb4d7abbb7bc8fbee1e8b9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:39.880403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.880844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.881184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:39.902717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:39.903947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:39.973644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 923/1836] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:39.979804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:39.980093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:39.980371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.001942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:40.003248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:40.012400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 914/1837] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:40.018020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.029612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792584-fa2c-45d8-8892-17dadddc51ec None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.034318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.034937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.035222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.035497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.036183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:40.036497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.038298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.050985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 924/1838] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 21:55:40.056052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.065537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be82b2-0540-451f-be98-c0a20b5a7044 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.070927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.071528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.071802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.072070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.072703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.074638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.103042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 915/1839] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:40 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:40.107453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.117323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8e6c-cac7-463a-91a0-8d984fceca61 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.173481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.174120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.174403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.174680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.175297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.177241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.198613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 925/1840] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:40 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:40.202765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.213579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f027-3eff-46d2-8a88-011ba9be27b6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.218196 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.218794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.219071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.219341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.219978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.221632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.231400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 916/1841] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:40.235417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.245127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed61842-d94b-4704-9e4a-9153d7537cf4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.249987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.250578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.250862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.251132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.251767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.253344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250349b-737e-4272-a5eb-3e7030085fd0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.264428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 926/1842] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:40.268035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.278658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45172f3-b96e-406f-afaa-301ea77a4b0f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.283240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.283828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.284107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.284380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] PATH_INFO: `/v3/projects/3c923a5e92054db2b7205218b04ecde6/users/bd71c9075347490983d1e6d4e98a1f7b/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.299083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c923a5e92054db2b7205218b04ecde6, user_id=bd71c9075347490983d1e6d4e98a1f7b, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.301837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.314284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 917/1843] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:40 2022] PUT /identity/v3/projects/3c923a5e92054db2b7205218b04ecde6/users/bd71c9075347490983d1e6d4e98a1f7b/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:40.325084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.325375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.325682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.343260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:40.358838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] MFA Rules not processed for user `bd71c9075347490983d1e6d4e98a1f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:40.360153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:40.435448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 927/1844] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:40.440055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.534674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327cf7-c656-41d7-b659-e68525ede01c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.540594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.541628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.541802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.542072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.542757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.544703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.563632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:40.585652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 918/1845] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:40 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:40.585892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.597181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e52da2-7534-47e9-986b-6f858d0d61bb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.677949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.678611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.678900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.679187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.680269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.682285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.714869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 928/1846] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:40 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:55:40.719052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.741031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94975378-c020-4cfe-8a2b-d503c1470302 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.746632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.747400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.747788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.748171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.749059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.751015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.764849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 919/1847] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:40.770195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.782234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74262cd-2f97-4b45-8ce4-c329f1c8a113 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:40.788218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:40.788952 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:40.789287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:40.789636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] PATH_INFO: `/v3/projects/7a958868df71447e9ccbd485fe490d58/users/7949e26330974c4abbfa236ecb5879ee/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:40.808985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a958868df71447e9ccbd485fe490d58, user_id=7949e26330974c4abbfa236ecb5879ee, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:40.811772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:40.841574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 929/1848] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:40 2022] PUT /identity/v3/projects/7a958868df71447e9ccbd485fe490d58/users/7949e26330974c4abbfa236ecb5879ee/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:40.845708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:40.867095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6deda9-662f-4df6-9c15-fa023d988e7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.006364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.007015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.007312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.007573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.008284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.010055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.033832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 920/1849] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:41.038665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:41.052238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717fd64-2f23-4c75-9117-344987e090a5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.053137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:41.055198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.059009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.060357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.060357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.060606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] PATH_INFO: `/v3/projects/7a958868df71447e9ccbd485fe490d58/users/7949e26330974c4abbfa236ecb5879ee/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.064175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a958868df71447e9ccbd485fe490d58, user_id=7949e26330974c4abbfa236ecb5879ee, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.067773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fad68-2fca-4829-92f8-9494a64b0ed2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.070552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbbd09-d547-4f43-a1ab-db6069554ea1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.075549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.076205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.076486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.076760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] PATH_INFO: `/v3/users/21001b5ea58942b0bcd0c4411f895e81` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.077969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21001b5ea58942b0bcd0c4411f895e81)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.080400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.094668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 930/1850] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:41 2022] PUT /identity/v3/projects/7a958868df71447e9ccbd485fe490d58/users/7949e26330974c4abbfa236ecb5879ee/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:41.097752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.099035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.099484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.161907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] MFA Rules not processed for user `7949e26330974c4abbfa236ecb5879ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:41.163505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.175709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21001b5ea58942b0bcd0c4411f895e81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.178176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21001b5ea58942b0bcd0c4411f895e81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.178969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21001b5ea58942b0bcd0c4411f895e81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.219563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:41.221269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21001b5ea58942b0bcd0c4411f895e81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.240375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ce0e43e7-16a4-42d2-bfca-ef2c2124262e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21001b5ea58942b0bcd0c4411f895e81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.274164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 921/1851] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:41 2022] DELETE /identity/v3/users/21001b5ea58942b0bcd0c4411f895e81 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:41.286834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.287034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.287814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.320831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 931/1852] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:41.333090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:41.336379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.343656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:41.346371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.434816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a79d18-2eaf-46ae-9934-a6a6eda65d77 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.441340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:41.462176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 922/1853] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:41.476830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:41.479551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.518753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.520823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.521316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.521422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.612566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd848-a9da-4eb2-89ea-07810f191207 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.618420 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.619349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.619752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.620160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.649117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.651970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.668849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 923/1854] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:41.681303 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.682801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84c56a-8a2c-4b08-abbb-99a732e70527 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.696196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 932/1855] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:41.758999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:41.760937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:41.780940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99990647-316d-4d58-b281-74fd88f87e00 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:41.794273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:41.795000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.795368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.795652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] PATH_INFO: `/v3/users/bd71c9075347490983d1e6d4e98a1f7b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:41.797422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd71c9075347490983d1e6d4e98a1f7b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:41.801863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:41.890395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd71c9075347490983d1e6d4e98a1f7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.892620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd71c9075347490983d1e6d4e98a1f7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.893919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd71c9075347490983d1e6d4e98a1f7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.922921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd71c9075347490983d1e6d4e98a1f7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.940136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd71c9075347490983d1e6d4e98a1f7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:41.969687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 924/1856] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:41 2022] DELETE /identity/v3/users/bd71c9075347490983d1e6d4e98a1f7b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:41.971332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:41.971724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:41.972100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.018644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:42.020976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.137132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:42.181522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 933/1857] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:42.190043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.192838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.275611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647630-298b-4080-ae8f-f999357cefb9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.279258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.280131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.280413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.280683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.291506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.292949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.313780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 925/1858] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:42.342097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.344696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.363044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc8df4-c395-459c-ba74-c91fca00503a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.377963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.378872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.379166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.379544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] PATH_INFO: `/v3/projects/a738d5c28b0f41c7a042a952fd28a797` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.382945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a738d5c28b0f41c7a042a952fd28a797)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.387516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.442676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a738d5c28b0f41c7a042a952fd28a797', 'request_id': 'req-a26c0d00-99ae-4400-9861-5e35ef5f9799'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.450949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a738d5c28b0f41c7a042a952fd28a797', 'request_id': 'req-a26c0d00-99ae-4400-9861-5e35ef5f9799'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.456886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a738d5c28b0f41c7a042a952fd28a797', 'request_id': 'req-a26c0d00-99ae-4400-9861-5e35ef5f9799'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.468462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] The token cache is being invalidate because project a738d5c28b0f41c7a042a952fd28a797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:42.468753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.469814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.471360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 934/1859] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:42 2022] DELETE /identity/v3/projects/a738d5c28b0f41c7a042a952fd28a797 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:42.475784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.486979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3eb4-a290-426b-9b3a-46c7c5d39d2e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.495691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:42.537100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.538661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.600241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.600826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.601097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.601395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] PATH_INFO: `/v3/users/aca9c3d145114468849f521772a7035c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.609762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca9c3d145114468849f521772a7035c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.612004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.626564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c0d00-99ae-4400-9861-5e35ef5f9799 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.633523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.634221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.634504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.634780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] PATH_INFO: `/v3/users/2027d81390a14f5db200e21058a04103` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.635938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2027d81390a14f5db200e21058a04103)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.639033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.671314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca9c3d145114468849f521772a7035c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.672032 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca9c3d145114468849f521772a7035c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.672765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca9c3d145114468849f521772a7035c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.686156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca9c3d145114468849f521772a7035c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.693017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2027d81390a14f5db200e21058a04103'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.700014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca9c3d145114468849f521772a7035c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.700646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2027d81390a14f5db200e21058a04103'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.709274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:42.716806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2027d81390a14f5db200e21058a04103'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.718466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 926/1860] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:42 2022] DELETE /identity/v3/users/aca9c3d145114468849f521772a7035c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:42.719540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.721057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.728789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2027d81390a14f5db200e21058a04103'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.729453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9048afe-6da2-4da8-bf22-d845a4b470b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2027d81390a14f5db200e21058a04103'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:42.736729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 935/1861] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:42 2022] DELETE /identity/v3/users/2027d81390a14f5db200e21058a04103 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:42.741949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.742352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.742628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.763495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:42.764886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.799845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b1ea2-8dcf-40fe-8c7f-1a0bd2df1aa9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.846314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:42.853589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 936/1862] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:42.867947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:42.869544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:42.901197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.902203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.902473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.902754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] PATH_INFO: `/v3/projects/3c923a5e92054db2b7205218b04ecde6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.905632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c923a5e92054db2b7205218b04ecde6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.907620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.958882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7947-02ef-4566-afa4-decec5a5d35b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:42.964220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:42.964220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:42.964220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:42.964513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:42.975392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:42.976791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:42.990558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 937/1863] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:43.002501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c923a5e92054db2b7205218b04ecde6', 'request_id': 'req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.008729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c923a5e92054db2b7205218b04ecde6', 'request_id': 'req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.020810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c923a5e92054db2b7205218b04ecde6', 'request_id': 'req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.034116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] The token cache is being invalidate because project 3c923a5e92054db2b7205218b04ecde6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:43.034398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.035181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.038072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 927/1864] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:42 2022] DELETE /identity/v3/projects/3c923a5e92054db2b7205218b04ecde6 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:43.042719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:43.044139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:43.067009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6ede-b6a7-481a-b894-d7ffc611d967 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:43.176524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:43.179254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:43.212879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:43.213549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:43.213877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:43.214146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] PATH_INFO: `/v3/users/7949e26330974c4abbfa236ecb5879ee` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:43.216148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7949e26330974c4abbfa236ecb5879ee)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:43.218143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:43.289038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6774-eb0c-4473-a7f6-262f7b4b82bd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:43.297186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:43.298090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:43.298371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:43.298738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] PATH_INFO: `/v3/projects/63e11634726f45749af72b1e592de0dd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:43.300960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e11634726f45749af72b1e592de0dd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:43.304544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:43.347026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:43.402956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7949e26330974c4abbfa236ecb5879ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.421440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7949e26330974c4abbfa236ecb5879ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.455980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7949e26330974c4abbfa236ecb5879ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.469740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e11634726f45749af72b1e592de0dd', 'request_id': 'req-7b06dce4-1d49-4968-8f52-157ab1171299'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.475548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7949e26330974c4abbfa236ecb5879ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.476832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7949e26330974c4abbfa236ecb5879ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.500898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e11634726f45749af72b1e592de0dd', 'request_id': 'req-7b06dce4-1d49-4968-8f52-157ab1171299'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.501547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 938/1865] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:43 2022] DELETE /identity/v3/users/7949e26330974c4abbfa236ecb5879ee => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:43.553698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e11634726f45749af72b1e592de0dd', 'request_id': 'req-7b06dce4-1d49-4968-8f52-157ab1171299'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.564172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:43.565644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:43.566597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:43.577276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] The token cache is being invalidate because project 63e11634726f45749af72b1e592de0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:43.578014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.580254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b06dce4-1d49-4968-8f52-157ab1171299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.588572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 928/1866] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:43 2022] DELETE /identity/v3/projects/63e11634726f45749af72b1e592de0dd => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:43.593538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d98d7-da4b-4bcd-b21a-318bd5ee6792 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:43.596803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d98d7-da4b-4bcd-b21a-318bd5ee6792 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:43.597395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d98d7-da4b-4bcd-b21a-318bd5ee6792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:43.663407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b02d98d7-da4b-4bcd-b21a-318bd5ee6792 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:43.666360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b02d98d7-da4b-4bcd-b21a-318bd5ee6792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:43.685272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e437a-8102-4251-90cf-1b1fa29956cf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:43.690210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 929/1867] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:43.698344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:43.700585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:43.701091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:43.734162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:43.734946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:43.735231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:43.735506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] PATH_INFO: `/v3/projects/f8dfd1ab64c340fe94d34f6f745717a8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:43.736824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dfd1ab64c340fe94d34f6f745717a8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:43.739909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:43.772597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:43.776366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:43.824434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 930/1868] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:43.829641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:43.846518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dfd1ab64c340fe94d34f6f745717a8', 'request_id': 'req-6ddcc72f-1273-46f2-a850-beff7d4f7fab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.857532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483a39f-3b5a-4b55-a95d-2f5affbe8cf0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:43.857919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dfd1ab64c340fe94d34f6f745717a8', 'request_id': 'req-6ddcc72f-1273-46f2-a850-beff7d4f7fab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.866854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dfd1ab64c340fe94d34f6f745717a8', 'request_id': 'req-6ddcc72f-1273-46f2-a850-beff7d4f7fab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.897152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] The token cache is being invalidate because project f8dfd1ab64c340fe94d34f6f745717a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:43.897471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.898263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:43.905179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 939/1869] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:43 2022] DELETE /identity/v3/projects/f8dfd1ab64c340fe94d34f6f745717a8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:43.906737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:43.908883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.009171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.009992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.010387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.010773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.011515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:44.011917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.013916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.033369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 931/1870] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:44.033765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:44.035558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.038506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.057198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcfe5e-9e17-4a38-8aca-c8e7cc4b70d7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.063820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.065463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.065906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.066251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.067528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.070727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.071536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc72f-1273-46f2-a850-beff7d4f7fab admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.089560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.090517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.090795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.091205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] PATH_INFO: `/v3/users/a87adc270cd245d78bf8544ab8a7dfc2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.092510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87adc270cd245d78bf8544ab8a7dfc2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.094808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.140330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 932/1871] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:44 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:44.142186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.146325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.186339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87adc270cd245d78bf8544ab8a7dfc2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.208770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87adc270cd245d78bf8544ab8a7dfc2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.236456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d109a-046c-4a1a-b48a-3bf1ec386d27 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.237663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87adc270cd245d78bf8544ab8a7dfc2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.246631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.254215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.254215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.254215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] PATH_INFO: `/v3/users/d461b953c9ee4901a1c144882d99da67` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.254215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d461b953c9ee4901a1c144882d99da67)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.254215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.266970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87adc270cd245d78bf8544ab8a7dfc2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.266970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87adc270cd245d78bf8544ab8a7dfc2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.276509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 940/1872] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:43 2022] DELETE /identity/v3/users/a87adc270cd245d78bf8544ab8a7dfc2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:44.286571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.289319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.375906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd461b953c9ee4901a1c144882d99da67'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.376916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd461b953c9ee4901a1c144882d99da67'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.377876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd461b953c9ee4901a1c144882d99da67'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.392066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:44.394197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ea53-ed6a-4335-9ae0-f03b2c40f305 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.399163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd461b953c9ee4901a1c144882d99da67'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.400328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.400965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.401236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.401855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.402203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.404729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.409191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07bbc35d-a472-4ac5-b7f5-bfad2f7189d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd461b953c9ee4901a1c144882d99da67'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.431332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 933/1873] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:44 2022] DELETE /identity/v3/users/d461b953c9ee4901a1c144882d99da67 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:44.436584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.436940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.437341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.465967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:44.467252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.479529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 941/1874] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:44 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:44.483406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.485886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.500275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949724b-3b8a-42b0-b99e-8222643ba6fb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.547128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 934/1875] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:44.555185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.558839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.571398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:44.605750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.607170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.608214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.608537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.609151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.611258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.629319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 942/1876] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:44.631251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.644873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593529-18f6-423c-b9a1-c53c3d1d8bab admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.654378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea402f09-8b45-4984-88db-b89a00e023b5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.659067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.659962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.660339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.660877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.661219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.661880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.662162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.662436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.663644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.666743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.678691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 943/1877] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:44.681361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.681704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.682860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.683536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.700739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde2cb-4d73-42f0-89af-5f0a678c8317 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.705809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.706453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.706738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.707017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] PATH_INFO: `/v3/projects/7a958868df71447e9ccbd485fe490d58` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.708234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a958868df71447e9ccbd485fe490d58)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.710201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.727030 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 935/1878] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:44.727924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:44.742157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955871-a176-47f4-b8d1-01e393750cc1 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:44.747230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:44.747937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.748219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.748492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] PATH_INFO: `/v3/projects/7d8c56658d75461fa91c8a393019aee5/users/bce16ba58a534dff8f132aec1e3f8999/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.770453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8c56658d75461fa91c8a393019aee5, user_id=bce16ba58a534dff8f132aec1e3f8999, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:44.776130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82844f-5c5e-4180-a939-ffd14c6f6c03 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:44.783469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a958868df71447e9ccbd485fe490d58', 'request_id': 'req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.793331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a958868df71447e9ccbd485fe490d58', 'request_id': 'req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.802298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 936/1879] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:44 2022] PUT /identity/v3/projects/7d8c56658d75461fa91c8a393019aee5/users/bce16ba58a534dff8f132aec1e3f8999/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:44.813319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073db10-e459-4487-9a32-920d506106f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.813319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073db10-e459-4487-9a32-920d506106f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.813319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073db10-e459-4487-9a32-920d506106f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.818996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a958868df71447e9ccbd485fe490d58', 'request_id': 'req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.836113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] The token cache is being invalidate because project 7a958868df71447e9ccbd485fe490d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:44.836406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.837145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-946d2cf2-70ee-422a-b452-2f3a8b0cbcf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:44.842514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 944/1880] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:44 2022] DELETE /identity/v3/projects/7a958868df71447e9ccbd485fe490d58 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:44.844295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:44.844730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:44.845281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:44.864133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3073db10-e459-4487-9a32-920d506106f7 None None] MFA Rules not processed for user `bce16ba58a534dff8f132aec1e3f8999`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:44.865998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3073db10-e459-4487-9a32-920d506106f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.894053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:44.897525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:44.939759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:44.994861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 937/1881] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:44.998533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 945/1882] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:44.999522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073db10-e459-4487-9a32-920d506106f7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.007439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3073db10-e459-4487-9a32-920d506106f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.007709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.009397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.041524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073db10-e459-4487-9a32-920d506106f7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.047849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.048532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.048846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.049068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] PATH_INFO: `/v3/projects/ca496a297a114f2da76e37b63e7500af` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.050358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca496a297a114f2da76e37b63e7500af)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.052945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.088603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:45.115053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca496a297a114f2da76e37b63e7500af', 'request_id': 'req-976e95eb-467e-4e7e-b348-042c865bdfb3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.121227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8efc4-ad0d-4346-a9a1-c0ec762f3fa1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.122450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca496a297a114f2da76e37b63e7500af', 'request_id': 'req-976e95eb-467e-4e7e-b348-042c865bdfb3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.133495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca496a297a114f2da76e37b63e7500af', 'request_id': 'req-976e95eb-467e-4e7e-b348-042c865bdfb3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.161536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] The token cache is being invalidate because project ca496a297a114f2da76e37b63e7500af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:45.161536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.161536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-976e95eb-467e-4e7e-b348-042c865bdfb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.170162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 938/1883] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:44 2022] DELETE /identity/v3/projects/ca496a297a114f2da76e37b63e7500af => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:45.173701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.173950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.174234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.199088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.199804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.200153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.200704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.203408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.221521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:45.222808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.272336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.274251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58973634-bfc4-4ab2-8f5f-e4393bfef6f2 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.284430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 946/1884] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:45.286564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.286749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.286918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.294064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 939/1885] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:45.299950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.301956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.315578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:45.317137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.318098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3d179-06d4-4b57-a5dc-119c6ed68c81 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.324647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.325392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.325737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.326204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] PATH_INFO: `/v3/projects/e2e1b244604c4e698620022e4265b028` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.327055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 947/1886] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:45.327587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e1b244604c4e698620022e4265b028)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.328033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.330046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.342751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccf22a-9150-4bcd-9d6b-007cd67b4fbc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.352404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.353515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.353818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.354091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.354893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:45.355202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.357030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294be01-e351-4e06-80e1-784bc5cdf1d1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.377556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 948/1887] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:45.379227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.379804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.380296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.405539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:45.407215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.450977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e1b244604c4e698620022e4265b028', 'request_id': 'req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.470417 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e1b244604c4e698620022e4265b028', 'request_id': 'req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.480615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:45.491194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e1b244604c4e698620022e4265b028', 'request_id': 'req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.517790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] The token cache is being invalidate because project e2e1b244604c4e698620022e4265b028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:45.518114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.518605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 949/1888] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:45.519628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.522091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.529691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 940/1889] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:45 2022] DELETE /identity/v3/projects/e2e1b244604c4e698620022e4265b028 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:45.530635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.530947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaa0f8-d069-4374-b028-0b888162afc1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.534187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.547308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.547936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.548216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.548486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.549148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.551065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.571585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd29d5-8cc5-4194-ae08-a7480c5cf9d1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.580505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.581582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.581949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.582382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.584880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:45.586321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.588841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.603265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 950/1890] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:45.605233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.608264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.615278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 941/1891] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:45.615278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.619072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:45.621323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:45.643896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ce266-73ed-4edb-8d2e-65ed67a770f8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.707978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256216a4-b247-43dd-b741-8e838fab6fa8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.722460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.723637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.724091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.724638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] PATH_INFO: `/v3/users/bf5a03e34bd54dbca854b28020da55d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.730792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5a03e34bd54dbca854b28020da55d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.733474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.743597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.744546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.744986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.745653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.746582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.748923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.775137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 942/1892] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:45.777450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.792908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1cec-20d0-4385-9c1f-bf127354f9a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:45.799216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:45.800048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:45.801962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:45.802255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:45.802897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:45.804828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:45.814301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5a03e34bd54dbca854b28020da55d2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.822755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5a03e34bd54dbca854b28020da55d2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.849826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5a03e34bd54dbca854b28020da55d2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.863380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 943/1893] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:45.865287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.881725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5a03e34bd54dbca854b28020da55d2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.883352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5a03e34bd54dbca854b28020da55d2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:45.899096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 951/1894] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:45 2022] DELETE /identity/v3/users/bf5a03e34bd54dbca854b28020da55d2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:45.900439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:45.902252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:46.040252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f90ff-7d65-447b-96cb-35186bacd843 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.051962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.052195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb43689-ee4c-4b6a-afd1-7d0f6acd5a2e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.052850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.053375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.053820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.054704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.056438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.059467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.060111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.060728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.060819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.061710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.063701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.079172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 944/1895] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:46.084204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.090914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:46.094928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 952/1896] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:45 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:46.102722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.115220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49653226-5e6e-4b94-9eb3-0cc5595fb661 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.121339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.122115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.122404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.122684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.124005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.125692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.129124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d1391-7061-4cd6-852f-8dc6abb46e5d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.132324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:46.137628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.138250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.138531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.139411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.140354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.142028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.144730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 953/1897] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:46.146833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.155203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 945/1898] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:46.160086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.167550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fe6e5-a697-4a21-957b-dfa55a90b48e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.182016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.183464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.184342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.185412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.186556 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.190574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.197143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a67a-6e1b-42a1-985a-fb52dd16511d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.218568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 954/1899] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:46.220539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.227854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.231441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.232134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.232134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] PATH_INFO: `/v3/projects/1705b638bcc54de382a0b050206ba1be/users/efe24c06f8d94a09b7628054e48ae919/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.235558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecb948-59a6-4930-8fe3-9685d194b9b7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.244992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.246776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.247741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.248214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] PATH_INFO: `/v3/projects/6a2ea4b9f3da4fb1bba508d806d6c303/users/0f6742f57ec042e683af7f167719600f/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.288388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1705b638bcc54de382a0b050206ba1be, user_id=efe24c06f8d94a09b7628054e48ae919, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.291490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c46a73-89e4-41d0-90b7-07dff3a98142 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.312510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2ea4b9f3da4fb1bba508d806d6c303, user_id=0f6742f57ec042e683af7f167719600f, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.313894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 946/1900] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:46 2022] PUT /identity/v3/projects/1705b638bcc54de382a0b050206ba1be/users/efe24c06f8d94a09b7628054e48ae919/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:46.321180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b764d-4c84-4e09-bf38-b15ce6d8ad7d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.325961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.326397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.326753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.345390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:46.362249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 955/1901] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:46 2022] PUT /identity/v3/projects/6a2ea4b9f3da4fb1bba508d806d6c303/users/0f6742f57ec042e683af7f167719600f/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:46.372724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] MFA Rules not processed for user `efe24c06f8d94a09b7628054e48ae919`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:46.374034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:46.375845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.376489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.377064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.426977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] MFA Rules not processed for user `0f6742f57ec042e683af7f167719600f`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:46.428892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:46.480529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 947/1902] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:46.481677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.521760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 956/1903] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:46.526792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.597044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca37c8-069e-4acf-843b-335090803082 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.621844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.625161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.626781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.627341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.628311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.634780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.717788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f468d4-9a7e-4243-935b-d2a52057ebf2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.724295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.725235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.725793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.726304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.727212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.729790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.736238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 948/1904] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/projects => generated 387 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:46.737206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.740619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:46.814168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 957/1905] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/projects => generated 389 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:46.815573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.870246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c6fb-c916-4289-8ac3-78123b4090dd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.885696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:46.917338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:46.920940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb28cb3-7138-4066-88fc-4bfd132cd7ca admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:46.930764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.932927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.933271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.933656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.934435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.937224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.982788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 958/1906] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:46.983293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:46.983840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:46.984311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:46.984846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:46.985327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] PATH_INFO: `/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:46.986751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d7ac02eb4d4bdfbaf4a1e94b1da51b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:46.989516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:46.997816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0856-d229-4060-a18c-f6e00b8fe7a5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.013841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.013841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.013841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.013841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.014226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.014226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.042601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 959/1907] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:46 2022] POST /identity/v3/users => generated 301 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:47.043581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.054525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b1ba3-355f-49d7-9474-481b87f64138 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.059577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d7ac02eb4d4bdfbaf4a1e94b1da51b', 'request_id': 'req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:47.066207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d7ac02eb4d4bdfbaf4a1e94b1da51b', 'request_id': 'req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:47.074904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d7ac02eb4d4bdfbaf4a1e94b1da51b', 'request_id': 'req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:47.090179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] The token cache is being invalidate because project 67d7ac02eb4d4bdfbaf4a1e94b1da51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:47.090513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:47.091628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:47.094193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 949/1908] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:46 2022] DELETE /identity/v3/projects/67d7ac02eb4d4bdfbaf4a1e94b1da51b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:47.095566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.099263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.130001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.130629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.130961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.131243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.131897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.134317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf606bf-be05-4a8c-87e2-9bc3ed836137 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.145070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 960/1909] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:47.146423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.146712 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.147047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.177463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:47.178766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.190202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 961/1910] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:47.191103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.192940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.201870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a4cc-f159-4a02-8eb8-9a2801bf3848 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.207263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58768a17-3c25-479c-9869-bf6bd0aa6a1e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.208279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.209869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.209869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.209869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.212769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.212769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a80d4-db80-4104-adb0-8df6e1882869 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.218590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.219257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.219572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.219854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] PATH_INFO: `/v3/projects/13c4d4d4a2384434bdc9c341aead2507/users/531c4ed475a740dfb3114333e1a35281/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.225723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 950/1911] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:47.225723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.225723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.227160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.243667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c4d4d4a2384434bdc9c341aead2507, user_id=531c4ed475a740dfb3114333e1a35281, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.246438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.247621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:47.249327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.261645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 951/1912] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:47.265542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.268429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 962/1913] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:47 2022] PUT /identity/v3/projects/13c4d4d4a2384434bdc9c341aead2507/users/531c4ed475a740dfb3114333e1a35281/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:47.269351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.276654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97248fd8-a008-456e-a105-1f35fec12697 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.344418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:47.355865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064415e1-91f0-4fbc-9cb8-675617879c42 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.359545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.361213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.361329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.362015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.362174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.362327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] PATH_INFO: `/v3/projects/33eaffc3db2e4174be9412106e90776b/users/22f0c51543fc4d65b7a2684c372e20ba/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.362427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.362616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.363260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:47.363566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.365230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.380933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 963/1914] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:47.380933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.386982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eaffc3db2e4174be9412106e90776b, user_id=22f0c51543fc4d65b7a2684c372e20ba, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.389944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.392831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec37b60-b436-467e-9d3a-9826d5645f4e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.399054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.399677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.399960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.400234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.400876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.402653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.405808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 952/1915] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:47 2022] PUT /identity/v3/projects/33eaffc3db2e4174be9412106e90776b/users/22f0c51543fc4d65b7a2684c372e20ba/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:47.408808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.409832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:47.418734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 964/1916] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:47.421153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.431422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035faaab-1ead-4aed-ba0b-0b2780249d1a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.492547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec962-c9d3-4195-9c5c-b14065fb3b8b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.499516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.500149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.500434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.500711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.501347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.503321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.505355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.506437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.506880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.507319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.508345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.510452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.525655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 965/1917] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:47.526770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.540143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c5c3-5062-4947-9ecd-ac99a16b868a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.543642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 953/1918] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:47.543642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.620865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.621691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.622077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.622269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca322-a323-4e0c-a76f-7dc16ff2fc0e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.622370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] PATH_INFO: `/v3/projects/13c4d4d4a2384434bdc9c341aead2507/users/531c4ed475a740dfb3114333e1a35281/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.624240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c4d4d4a2384434bdc9c341aead2507, user_id=531c4ed475a740dfb3114333e1a35281, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.627010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.627129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.627680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.627961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.628239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] PATH_INFO: `/v3/projects/33eaffc3db2e4174be9412106e90776b/users/22f0c51543fc4d65b7a2684c372e20ba/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.634648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eaffc3db2e4174be9412106e90776b, user_id=22f0c51543fc4d65b7a2684c372e20ba, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.639327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4486b64-b129-4c72-81d5-f62380741c66 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.645421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 966/1919] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:47 2022] PUT /identity/v3/projects/13c4d4d4a2384434bdc9c341aead2507/users/531c4ed475a740dfb3114333e1a35281/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:47.651371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.657704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 954/1920] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:47 2022] PUT /identity/v3/projects/33eaffc3db2e4174be9412106e90776b/users/22f0c51543fc4d65b7a2684c372e20ba/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:47.659166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efed2f-a2e2-444e-a043-7f2d0f3b3d83 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.659456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efed2f-a2e2-444e-a043-7f2d0f3b3d83 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.660113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efed2f-a2e2-444e-a043-7f2d0f3b3d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.710980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-76efed2f-a2e2-444e-a043-7f2d0f3b3d83 None None] MFA Rules not processed for user `531c4ed475a740dfb3114333e1a35281`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:47.712226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-76efed2f-a2e2-444e-a043-7f2d0f3b3d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.782440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157df3d-37f3-4c40-abfb-a36a13378a7c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:47.787303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:47.826735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 955/1921] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:47.828195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.828485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.829103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.843056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:47.874155 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] MFA Rules not processed for user `22f0c51543fc4d65b7a2684c372e20ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:47.876095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.904680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:47.905443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:47.905761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:47.906704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:47.907348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:47.909976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:47.950257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 967/1922] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/users => generated 282 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:47.951400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:47.953333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:47.970000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 956/1923] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:47.994003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.031250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0a941-6b7c-4da0-bade-27fb1b8a5d99 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.038413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96af1be-a30e-446a-9159-a6b64ebc4c8a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.039558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.040193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.040476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.040756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.042455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.044172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.044591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.044797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.045078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.045350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.058283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.058999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 957/1924] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:48.060056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.063519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.063666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.081006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec8a80-9c94-4edb-bde9-127795089c0f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.088066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.089916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.090412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.090903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.092211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 968/1925] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:48.093734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.118498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d2-239d-4d4b-9f19-768650866789 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.135074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.135827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.136206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.136583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.141949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.144331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.145859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.148755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.171342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 969/1926] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:48.171342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.176461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 958/1927] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:48.177111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.192105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a58b0-846b-4e31-a4ee-57466f7ee27a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.196136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d846e-2bd7-48a0-b5c4-d0f662c24c0c service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.202601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.204805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.205369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.207010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] PATH_INFO: `/v3/projects/bea40f8c931c4cb4812d2c1994e74286/users/d258d896c3d04c62adc3ffeea0fec844/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.207132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.207132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.207132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.207132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] PATH_INFO: `/v3/users/efe24c06f8d94a09b7628054e48ae919` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.209684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe24c06f8d94a09b7628054e48ae919)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.211628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.254626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea40f8c931c4cb4812d2c1994e74286, user_id=d258d896c3d04c62adc3ffeea0fec844, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.262096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd11a1-5e91-4865-a6d2-4c9088d64077 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.319360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe24c06f8d94a09b7628054e48ae919'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.326891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 959/1928] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:48 2022] PUT /identity/v3/projects/bea40f8c931c4cb4812d2c1994e74286/users/d258d896c3d04c62adc3ffeea0fec844/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:48.342298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe24c06f8d94a09b7628054e48ae919'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.357511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.357875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.358234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.413843 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe24c06f8d94a09b7628054e48ae919'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.443113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe24c06f8d94a09b7628054e48ae919'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.443841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93f1cf44-81e7-4572-bacb-7276c7cb5d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe24c06f8d94a09b7628054e48ae919'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.454675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] MFA Rules not processed for user `d258d896c3d04c62adc3ffeea0fec844`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:48.455934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.458606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 970/1929] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:48 2022] DELETE /identity/v3/users/efe24c06f8d94a09b7628054e48ae919 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:48.486994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.486994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.486994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.551746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:48.552990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.576542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 960/1930] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:48.578535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.580919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.589151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:48.593201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:48.656612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 971/1931] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:48.656612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.656612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.670457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575ce4d-7ec1-45fd-9c8c-4b1cc80f40f7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.677644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.677644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.677644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.677644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.678332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.682282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.698355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600e143-6fa5-44a6-a53a-9504aba065ef None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.703797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.704969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.705710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.706226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] PATH_INFO: `/v3/users/0f6742f57ec042e683af7f167719600f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.707687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6742f57ec042e683af7f167719600f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.710175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.736905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 972/1932] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:48 2022] POST /identity/v3/projects => generated 363 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:48.739751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.748467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.780801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6742f57ec042e683af7f167719600f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.781774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6742f57ec042e683af7f167719600f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.782497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6742f57ec042e683af7f167719600f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.800519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6742f57ec042e683af7f167719600f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.810856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6742f57ec042e683af7f167719600f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:48.828059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 961/1933] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:48 2022] DELETE /identity/v3/users/0f6742f57ec042e683af7f167719600f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:48.829247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:48.832126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.840934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69755ec3-f227-41bd-bd4b-9cfb06553719 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.939903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.940594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.940805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.941081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.946874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:48.954680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db0f3-d32d-4f8a-9fb5-3afdbe62d2bc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:48.964504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:48.964996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.965163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.965425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.965727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:48.966369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:48.966827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1416b-95e5-4b7c-9a6a-3c7250799c96 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.968494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:48.980034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 973/1934] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:48.980710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:48.981325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:48.981325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.003572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 962/1935] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:48 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:49.008441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.011412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:49.012400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.026950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 974/1936] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:49.030352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b6b0-bb9a-4c6c-878d-cf29c3f6a457 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.035644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.036328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.036877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.037211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.038131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.038220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.039394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.039574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.046853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:49.057865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 963/1937] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:49.058081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc27e2-46b8-48c3-8b51-7bc268799d82 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.062683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.063382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.063772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.064066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.065506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.087851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65197-48ab-4b15-8872-c346d88ef685 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.091589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.093032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.099042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.100811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.101098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.101414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] PATH_INFO: `/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995/users/03ff1e389eaf452480df37d1e8af8417/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.121662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 975/1938] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:49.127803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9a5a3c2b7a42e7958d648dc1f61995, user_id=03ff1e389eaf452480df37d1e8af8417, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.132610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.154178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 964/1939] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:49 2022] PUT /identity/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995/users/03ff1e389eaf452480df37d1e8af8417/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:49.155503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.179709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc36132-2e45-458e-bb02-b5f0ba499717 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.242860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:49.270328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.270939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.271227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.271503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.272329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.274236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b621ce-303f-44bf-b7b0-279f69c2db0b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.289890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 976/1940] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:49.294410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.311808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0a2a2-ce66-4917-a659-776c697c64e9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.318374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.319224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.319577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.319927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] PATH_INFO: `/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995/users/03ff1e389eaf452480df37d1e8af8417/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.322240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9a5a3c2b7a42e7958d648dc1f61995, user_id=03ff1e389eaf452480df37d1e8af8417, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.325671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.344710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 965/1941] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:49 2022] PUT /identity/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995/users/03ff1e389eaf452480df37d1e8af8417/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:49.359154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542239cd-3b85-4514-9302-731c7092775b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.359154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542239cd-3b85-4514-9302-731c7092775b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.360096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542239cd-3b85-4514-9302-731c7092775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.415569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-542239cd-3b85-4514-9302-731c7092775b None None] MFA Rules not processed for user `03ff1e389eaf452480df37d1e8af8417`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:49.418152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-542239cd-3b85-4514-9302-731c7092775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.500433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.510213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.519916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 977/1942] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:49.534530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542239cd-3b85-4514-9302-731c7092775b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.536159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-542239cd-3b85-4514-9302-731c7092775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.665635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264aff1c-0cbc-4553-9e23-d2530ad2260c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.671240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542239cd-3b85-4514-9302-731c7092775b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.672191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.673224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.673623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.674009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] PATH_INFO: `/v3/projects/6a2ea4b9f3da4fb1bba508d806d6c303` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.675290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2ea4b9f3da4fb1bba508d806d6c303)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.676975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:49.677797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:49.678214 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:49.678633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:49.678874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.690887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:49.693292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:49.724050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 978/1943] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:49.821127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2ea4b9f3da4fb1bba508d806d6c303', 'request_id': 'req-5881e66c-8193-4478-8cf8-a7c9c506cee1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:49.825801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.827721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:49.851484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2ea4b9f3da4fb1bba508d806d6c303', 'request_id': 'req-5881e66c-8193-4478-8cf8-a7c9c506cee1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:49.868690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2ea4b9f3da4fb1bba508d806d6c303', 'request_id': 'req-5881e66c-8193-4478-8cf8-a7c9c506cee1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:49.899302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] The token cache is being invalidate because project 6a2ea4b9f3da4fb1bba508d806d6c303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:49.899693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:49.902289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:49.904921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 966/1944] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:49 2022] DELETE /identity/v3/projects/6a2ea4b9f3da4fb1bba508d806d6c303 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:49.905954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e17554-3166-4c38-b318-78ad35168236 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:49.908818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:49.921246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.022163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:50.064291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.065079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.065437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.065817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881e66c-8193-4478-8cf8-a7c9c506cee1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:50.065950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] PATH_INFO: `/v3/projects/1705b638bcc54de382a0b050206ba1be` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.067232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1705b638bcc54de382a0b050206ba1be)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.070094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:50.077311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.082020 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.082566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.083057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] PATH_INFO: `/v3/users/22f0c51543fc4d65b7a2684c372e20ba` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.085281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f0c51543fc4d65b7a2684c372e20ba)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.093763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:50.110332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:50.182190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f0c51543fc4d65b7a2684c372e20ba'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.182950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f0c51543fc4d65b7a2684c372e20ba'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.183570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f0c51543fc4d65b7a2684c372e20ba'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.198752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1705b638bcc54de382a0b050206ba1be', 'request_id': 'req-9c1ecfa5-9f14-4753-b87c-507362cc87fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.208682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f0c51543fc4d65b7a2684c372e20ba'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.208979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1705b638bcc54de382a0b050206ba1be', 'request_id': 'req-9c1ecfa5-9f14-4753-b87c-507362cc87fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.219121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1705b638bcc54de382a0b050206ba1be', 'request_id': 'req-9c1ecfa5-9f14-4753-b87c-507362cc87fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.222783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f0c51543fc4d65b7a2684c372e20ba'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.236880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] The token cache is being invalidate because project 1705b638bcc54de382a0b050206ba1be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:50.237205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.238187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.243954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 979/1945] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:49 2022] DELETE /identity/v3/projects/1705b638bcc54de382a0b050206ba1be => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:50.245296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:50.246832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.250538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 967/1946] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:49 2022] DELETE /identity/v3/users/22f0c51543fc4d65b7a2684c372e20ba => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:50.377279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ecfa5-9f14-4753-b87c-507362cc87fd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:50.477963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.479443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.480102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.480679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] PATH_INFO: `/v3/users/531c4ed475a740dfb3114333e1a35281` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.482899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=531c4ed475a740dfb3114333e1a35281)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.485623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:50.508042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:50.509488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.523320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e509-639c-423b-bb1c-c220c7697da2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:50.529482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.530322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.530671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.531194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] PATH_INFO: `/v3/users/d258d896c3d04c62adc3ffeea0fec844` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.532641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d258d896c3d04c62adc3ffeea0fec844)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.534841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:50.564919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531c4ed475a740dfb3114333e1a35281'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.572988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:50.577766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531c4ed475a740dfb3114333e1a35281'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.603817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531c4ed475a740dfb3114333e1a35281'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.605845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd258d896c3d04c62adc3ffeea0fec844'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.606770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd258d896c3d04c62adc3ffeea0fec844'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.607331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd258d896c3d04c62adc3ffeea0fec844'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.613422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531c4ed475a740dfb3114333e1a35281'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.614316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1c50599e-1261-4497-a4f1-7e6d1380b48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531c4ed475a740dfb3114333e1a35281'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.621100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 980/1947] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:50 2022] DELETE /identity/v3/users/531c4ed475a740dfb3114333e1a35281 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:50.626326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd258d896c3d04c62adc3ffeea0fec844'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.635216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd258d896c3d04c62adc3ffeea0fec844'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:50.658891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 968/1948] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:50 2022] DELETE /identity/v3/users/d258d896c3d04c62adc3ffeea0fec844 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:50.663526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.663713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.664027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.687011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:50.689800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.772230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 981/1949] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:50.804719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:50.810470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.880583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:50.905015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9e8b-9486-447d-a502-57d8683c351c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:50.911697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:50.914211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:50.924943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.925873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.925873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.926277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.932351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ada81-230e-4d0a-90c7-366c7173a0de None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:50.938071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:50.939147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:50.939282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:50.939671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] PATH_INFO: `/v3/projects/33eaffc3db2e4174be9412106e90776b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:50.941117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=33eaffc3db2e4174be9412106e90776b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.946322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:50.967101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:50.969173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020225d-d4a0-4caa-befa-b2f42d1e7b66 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.009897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 969/1950] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:51.032122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33eaffc3db2e4174be9412106e90776b', 'request_id': 'req-457ec4d6-e993-432f-a7eb-06c94b08b69e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.042152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33eaffc3db2e4174be9412106e90776b', 'request_id': 'req-457ec4d6-e993-432f-a7eb-06c94b08b69e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.049491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33eaffc3db2e4174be9412106e90776b', 'request_id': 'req-457ec4d6-e993-432f-a7eb-06c94b08b69e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.070430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] The token cache is being invalidate because project 33eaffc3db2e4174be9412106e90776b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:51.070430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.071231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.072956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 982/1951] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:50 2022] DELETE /identity/v3/projects/33eaffc3db2e4174be9412106e90776b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:51.077759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deac87-0d7f-42c7-81c2-759b506b0249 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.078079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deac87-0d7f-42c7-81c2-759b506b0249 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.078320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deac87-0d7f-42c7-81c2-759b506b0249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.105101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-16deac87-0d7f-42c7-81c2-759b506b0249 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:51.107780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-16deac87-0d7f-42c7-81c2-759b506b0249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.148555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.151287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.230049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 970/1952] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:51.240262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.240666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.240953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.265460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:51.267193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:51.267450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.296835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ec4d6-e993-432f-a7eb-06c94b08b69e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.302013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 971/1953] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:51.305954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.306373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.307161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.307534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.307903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] PATH_INFO: `/v3/projects/13c4d4d4a2384434bdc9c341aead2507` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.309509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c4d4d4a2384434bdc9c341aead2507)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.311918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.323409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4987-b63f-4516-ae3c-ac353d8556f8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.339341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.339985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.341456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.341784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.343832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:51.344653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.349817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.371176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 972/1954] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:51.382650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.396244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00d263-97e2-469f-86b3-b46436f27952 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.398505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c4d4d4a2384434bdc9c341aead2507', 'request_id': 'req-7dbe490f-1e69-4f4a-a677-8509b626a798'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.410424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c4d4d4a2384434bdc9c341aead2507', 'request_id': 'req-7dbe490f-1e69-4f4a-a677-8509b626a798'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.435186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c4d4d4a2384434bdc9c341aead2507', 'request_id': 'req-7dbe490f-1e69-4f4a-a677-8509b626a798'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.458228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] The token cache is being invalidate because project 13c4d4d4a2384434bdc9c341aead2507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:51.459050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.461037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.462626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 983/1955] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:51 2022] DELETE /identity/v3/projects/13c4d4d4a2384434bdc9c341aead2507 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:51.463599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.467537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.473985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.474769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.475382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.475382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.476286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.478106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1e64c-04b4-42b7-9e74-346b799ed458 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.506329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe490f-1e69-4f4a-a677-8509b626a798 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.514307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.514502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.514775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.518221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 973/1956] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:51.518791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] PATH_INFO: `/v3/projects/bea40f8c931c4cb4812d2c1994e74286` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.518879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.518879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.518879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.519076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea40f8c931c4cb4812d2c1994e74286)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.521548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.543063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:51.544459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.587384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea40f8c931c4cb4812d2c1994e74286', 'request_id': 'req-4895236b-1dad-4350-9555-13574ae16808'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.597551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea40f8c931c4cb4812d2c1994e74286', 'request_id': 'req-4895236b-1dad-4350-9555-13574ae16808'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.607502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea40f8c931c4cb4812d2c1994e74286', 'request_id': 'req-4895236b-1dad-4350-9555-13574ae16808'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.614664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:51.622662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] The token cache is being invalidate because project bea40f8c931c4cb4812d2c1994e74286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:51.623274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.623954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.625644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 984/1957] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:51 2022] DELETE /identity/v3/projects/bea40f8c931c4cb4812d2c1994e74286 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:51.626497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.627766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.629219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 974/1958] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:51.637380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.643800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.658170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd05bb-7262-4223-89b8-008df19b092e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.720431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895236b-1dad-4350-9555-13574ae16808 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.732028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.732884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.733231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.733632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.734362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.737387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.738100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.738210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe5cc2-71b1-4899-827c-1e4c825df179 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.738375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.738645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] PATH_INFO: `/v3/users/03ff1e389eaf452480df37d1e8af8417` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.739806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ff1e389eaf452480df37d1e8af8417)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.741773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.753870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:51.772097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 985/1959] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:51.774134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5530755-a099-4b35-b773-2346593a8648 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.774543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5530755-a099-4b35-b773-2346593a8648 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.774888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5530755-a099-4b35-b773-2346593a8648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.805167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f5530755-a099-4b35-b773-2346593a8648 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:51.806676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f5530755-a099-4b35-b773-2346593a8648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.815016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 986/1960] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:51.816322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5530755-a099-4b35-b773-2346593a8648 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.819931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ff1e389eaf452480df37d1e8af8417'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.821113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ff1e389eaf452480df37d1e8af8417'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.821997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ff1e389eaf452480df37d1e8af8417'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.828000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5530755-a099-4b35-b773-2346593a8648 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.835238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ff1e389eaf452480df37d1e8af8417'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.836823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.838360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.839207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.839696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.840827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.843521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ff1e389eaf452480df37d1e8af8417'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:51.843986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.856704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 987/1961] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:51.857857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.860424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 975/1962] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:51 2022] DELETE /identity/v3/users/03ff1e389eaf452480df37d1e8af8417 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:51.860554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.861482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.866306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:51.875482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30768b-b63a-4a8d-a47c-734a7fd04ea2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.944790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.945467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.946011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.946156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.946772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:51.947078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.948802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.962605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 988/1963] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:51.966706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:51.980449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78adaae-cb24-4c27-bdcf-fddbec7906e6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.982493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5846a4-703d-4e1b-86e2-63b11a885a13 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:51.987533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.988383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.989330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.989797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.990800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:51.994427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:51.995234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:51.995947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:51.996332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:51.997781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:51.998432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.003837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.033171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 976/1964] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:52.038836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.055767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fa3d-f848-4803-b3d6-9271df3988d7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.065572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 989/1965] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:51 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:52.073770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.074166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.074635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.075355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.075355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] PATH_INFO: `/v3/projects/0f06d27c2159414f88e9a803d8d0b86f/users/381fb3ca62a747bba4820d5e69637956/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.084177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:52.125653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f06d27c2159414f88e9a803d8d0b86f, user_id=381fb3ca62a747bba4820d5e69637956, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.131765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ab1c7-8d97-44c4-a899-2aa82f8f537a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.155747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 977/1966] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:52 2022] PUT /identity/v3/projects/0f06d27c2159414f88e9a803d8d0b86f/users/381fb3ca62a747bba4820d5e69637956/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:52.167661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.168347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.168909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.176883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2919d30-1e03-491e-ab4a-c7a48725c313 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.228460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] MFA Rules not processed for user `381fb3ca62a747bba4820d5e69637956`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:52.230215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.257880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:52.267940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.268569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.268853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.269125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.269941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.272063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.304045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 990/1967] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:52.304045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.304045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.315775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f491c-51d2-4d0e-b690-6a7b69eea85e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.320638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.321287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.321570 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.326568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] PATH_INFO: `/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.327990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9a5a3c2b7a42e7958d648dc1f61995)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.330032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.334138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 978/1968] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:52.334138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.352889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb716b5-f6a5-4434-978e-4ce86cfbc5de None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.363102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.364577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.364927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.365635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.366220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.368134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.381933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 979/1969] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:52.381933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.393001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d9a3-d5e1-4186-b7b1-f8b662a9bb39 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.405896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9a5a3c2b7a42e7958d648dc1f61995', 'request_id': 'req-47926765-cb75-4e8a-b9df-3ac2757a531f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:52.416467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9a5a3c2b7a42e7958d648dc1f61995', 'request_id': 'req-47926765-cb75-4e8a-b9df-3ac2757a531f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:52.423879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9a5a3c2b7a42e7958d648dc1f61995', 'request_id': 'req-47926765-cb75-4e8a-b9df-3ac2757a531f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:52.438680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] The token cache is being invalidate because project 4a9a5a3c2b7a42e7958d648dc1f61995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:52.438981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:52.439664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:52.441292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 991/1970] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:52 2022] DELETE /identity/v3/projects/4a9a5a3c2b7a42e7958d648dc1f61995 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:52.442151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.443490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.457861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.458484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.458764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.459042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.459667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.461589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80cf8-d20a-4784-af7f-cc4437280643 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.496931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 980/1971] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:52.498609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.499169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.499587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.524503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:52.525775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.533937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47926765-cb75-4e8a-b9df-3ac2757a531f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.581213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:52.614916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 981/1972] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:52.614916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.614916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.640403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.641533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.641533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.641786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.642329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.643941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94327636-2fe7-4938-ab5b-04f6b8f75145 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.653800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eced7c-ee26-444c-b829-73f71d305686 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.664344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.666426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.666878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.667400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.668612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.670228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 992/1973] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:52.671665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.671965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.672238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.672752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.711319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 982/1974] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:52.712352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.719412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:52.720666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.731539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 993/1975] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:52.732413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.737345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5aa876-dd7e-4f6d-bce6-6567b066d420 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.745822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb82a04-5f00-444d-8144-5568d101c08e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.747451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.748785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.749670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.750318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] PATH_INFO: `/v3/projects/07f729325cb84b048432af82515783b2/users/45526c7557e64259afbe75b3043eb4bb/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.751267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.751888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.752246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.752541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.753275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.755615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.757034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:52.772258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 994/1976] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:52.773309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.777075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f729325cb84b048432af82515783b2, user_id=45526c7557e64259afbe75b3043eb4bb, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.786845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8f142-0678-489a-8540-5ea90e663c71 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.787088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.792336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.792968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.793246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.793543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.794223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:55:52.794531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.796504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fdfa0-192d-47cd-bff5-d08514806522 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.805027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 983/1977] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:52 2022] PUT /identity/v3/projects/07f729325cb84b048432af82515783b2/users/45526c7557e64259afbe75b3043eb4bb/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:52.805752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.812205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 995/1978] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:55:52.814975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.816130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.817750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.850220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] MFA Rules not processed for user `45526c7557e64259afbe75b3043eb4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:52.851507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:52.880582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a21dff-b139-440f-8d74-95ee3ae900c8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:52.886132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:52.886802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:52.887084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:52.887360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] PATH_INFO: `/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0/users/78417e7f939a444ea46e418d47012246/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:52.904597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd78f6d10d347d1ac26aee14399c1f0, user_id=78417e7f939a444ea46e418d47012246, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:52.907663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:52.930338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 984/1979] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:52 2022] PUT /identity/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0/users/78417e7f939a444ea46e418d47012246/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:52.931172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.938684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 996/1980] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:52.947173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:52.959671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776bd8f-bb40-45ff-b81b-988cd0817942 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.014596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8e362-76ba-4e5a-a583-47266d917433 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.019573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.021096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.021588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.022006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.022740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.024769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.036851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.037706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.037924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.038202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.038854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.040458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.055027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 997/1981] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:53.055027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.056099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 985/1982] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:52 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:53.057026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.064183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6a79a-9bd8-4218-a59c-2d83d2294832 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.080376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:53.139872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920f7dd-1db2-44ba-9486-d3d528b0d0a3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.142994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.143714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.144071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.144418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.145632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.145774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.146396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.146679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.146957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] PATH_INFO: `/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0/users/78417e7f939a444ea46e418d47012246/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.147759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abf8a2-2b4c-452f-8319-85e342f74ecd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.149039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd78f6d10d347d1ac26aee14399c1f0, user_id=78417e7f939a444ea46e418d47012246, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.151749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.163201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 986/1983] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:53 2022] PUT /identity/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0/users/78417e7f939a444ea46e418d47012246/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:53.164044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.180993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:53.225357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 998/1984] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:53.225357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.225357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.225357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.243297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638391bd-12cf-4717-8ab1-2fa41dc7662b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.258508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] MFA Rules not processed for user `78417e7f939a444ea46e418d47012246`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:53.259863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:53.312815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.313465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.313794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.314076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.314884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.317676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.350354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 987/1985] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:53.351184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.351690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 999/1986] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:53.357066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.364333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1643-d19d-40b2-b419-9ab22ba33259 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.370222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.370956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508e084-7c3d-4071-ba42-80a9c6a6fcad None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.371124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.371617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.371891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.372543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.374633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.375728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.376299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.376575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.376842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.377498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.379124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.394638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1000/1987] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:53.395532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.399047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:53.407876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 988/1988] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:53.407876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.419068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239118d4-2e0a-40e1-9d8f-8ae9c6a0668c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.423941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.424754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.425144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.425329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.426005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.428268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.438841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 989/1989] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:53.439641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.450320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c509-adfc-4548-a6ba-ddd86373ee17 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.455113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.455714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.455995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.456271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.456908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.458626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.464741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d80ec8-152b-4e68-bea7-d3a7daed2aec admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.468308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.468907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.469462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.469569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 990/1990] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:53.469698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.470233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.480583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0880-f782-4e3a-8739-0b7d367d7ff2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.482853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.484330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.485259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.485951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.486233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.486508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] PATH_INFO: `/v3/projects/38d790fc1f674e5f983f2806d962bc70/users/6292f50084be42cf999c00d6b66931c4/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.489319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:53.495461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1001/1991] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:53.496272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.505538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d790fc1f674e5f983f2806d962bc70, user_id=6292f50084be42cf999c00d6b66931c4, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.506903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d06b1c-c880-4eec-b151-e9b76d7add5a service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.508283 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20414e14-39c4-49af-bc6d-c07c680147eb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.511788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.512405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.512670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.512938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.513574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.515293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.530465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1002/1992] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:53.530679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 991/1993] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:53 2022] PUT /identity/v3/projects/38d790fc1f674e5f983f2806d962bc70/users/6292f50084be42cf999c00d6b66931c4/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:53.537987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.539815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.540138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.540455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.566591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289351d2-f26c-43c1-9ced-a9e10c4ef307 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.586501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] MFA Rules not processed for user `6292f50084be42cf999c00d6b66931c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:53.587782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:53.651721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.652307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.652592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.652906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] PATH_INFO: `/v3/projects/cef52abdf9a9485382412aabaa4a9091/users/85f412229d6c41ca886d53984b1a932b/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.655825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:53.674478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef52abdf9a9485382412aabaa4a9091, user_id=85f412229d6c41ca886d53984b1a932b, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.678220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.702033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1003/1994] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:53 2022] PUT /identity/v3/projects/cef52abdf9a9485382412aabaa4a9091/users/85f412229d6c41ca886d53984b1a932b/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:53.702669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.704556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 992/1995] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:53.713233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.728262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb76b7-4a97-46d9-ac91-39d91b4a8789 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.777902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252857-6967-4942-9853-b7b41a9fca28 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.782435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.783052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.783336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.783614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.786163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:53.796079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.796783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.797133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.797469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.804125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.804125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c002-ef7c-497e-b2d3-1cbd9d891663 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.831643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 993/1996] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/projects => generated 401 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:53.833158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.833540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.833895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.862989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] MFA Rules not processed for user `85f412229d6c41ca886d53984b1a932b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:53.864908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:53.867941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.873647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:53.883109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1004/1997] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:53.883979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.898363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272dc77-5067-409f-b61f-536198647c51 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:53.939190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:53.969746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 994/1998] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:53.972643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:53.989562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:53.990419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:53.990697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:53.991209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:53.992051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:53.994471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.021842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1005/1999] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/users => generated 307 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:54.025839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.036311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d8992-3b07-4052-a8c3-da06d4cc10e5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.043908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.044294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.044590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.044962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.045652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.047231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.073370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e60a7d-db88-4210-bce7-264f44c93749 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.077998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.078799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.079305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.079622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.080409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.084228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.093934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1006/2000] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:55:54.095875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.131041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f7a5-d4b2-40ed-a7f4-5cdbbcd6db22 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.136040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 995/2001] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:55:53 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:54.137021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.199433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:54.221061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d16a9-37ca-4fb2-9c28-6b3ef0d27658 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.227484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.228263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.228640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.229084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.229959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.232853 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.274515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 996/2002] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:55:54 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:55:54.285079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.305473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372fc10-de97-42de-b4ce-fecba77f8166 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.331539 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.331787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.333330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.333972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.335848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.340121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.345461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.346311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.346709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.347041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] PATH_INFO: `/v3/projects/82368c4e1901456f8e0da482a2869454/users/2bae74f6c08e4344ab23ce7d271284cf/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.376081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 997/2003] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:54.381053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.393930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82368c4e1901456f8e0da482a2869454, user_id=2bae74f6c08e4344ab23ce7d271284cf, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.398075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.430270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea1d11-83db-4deb-9738-0a716c1b1515 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.439386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1007/2004] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:54 2022] PUT /identity/v3/projects/82368c4e1901456f8e0da482a2869454/users/2bae74f6c08e4344ab23ce7d271284cf/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:54.446324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.495553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:54.511673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.512672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.512973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.513651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] PATH_INFO: `/v3/projects/3a67119e02b744e29dfe3ad1865798d2/users/9b8e858cf7da4d9faf8197b7a934a7c1/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.535120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a67119e02b744e29dfe3ad1865798d2, user_id=9b8e858cf7da4d9faf8197b7a934a7c1, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.539003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.548201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20567ede-29c9-46aa-b75c-4b839d10ad00 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.558103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.558929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 998/2005] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:54 2022] PUT /identity/v3/projects/3a67119e02b744e29dfe3ad1865798d2/users/9b8e858cf7da4d9faf8197b7a934a7c1/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:54.559436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.559947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.560561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.562646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.562859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.565246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.587919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1008/2006] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:54.593943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.607376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981577a-b425-4e9b-a21e-54ae85ff9da4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.672452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed907697-4463-4388-b4e8-b990e26cd3c5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.678388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.679092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.679443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.679787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.680982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.681175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.681431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.681920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.682275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] PATH_INFO: `/v3/projects/82368c4e1901456f8e0da482a2869454/users/2bae74f6c08e4344ab23ce7d271284cf/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.682439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.684389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82368c4e1901456f8e0da482a2869454, user_id=2bae74f6c08e4344ab23ce7d271284cf, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.687403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2b087-03e5-4ebb-9abf-0c7695643047 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.696453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 999/2007] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 21:55:54.701680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.715820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1009/2008] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:54 2022] PUT /identity/v3/projects/82368c4e1901456f8e0da482a2869454/users/2bae74f6c08e4344ab23ce7d271284cf/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:54.716039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba170c-29d1-4f80-a68b-e678417946de admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.724102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb4781-551f-4955-8903-62914d556908 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.724610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb4781-551f-4955-8903-62914d556908 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.725036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb4781-551f-4955-8903-62914d556908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.761569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dddb4781-551f-4955-8903-62914d556908 None None] MFA Rules not processed for user `2bae74f6c08e4344ab23ce7d271284cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:54.762885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dddb4781-551f-4955-8903-62914d556908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:54.800113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:54.801984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:54.803410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.803686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.804087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] PATH_INFO: `/v3/projects/3a67119e02b744e29dfe3ad1865798d2/users/9b8e858cf7da4d9faf8197b7a934a7c1/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.806963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a67119e02b744e29dfe3ad1865798d2, user_id=9b8e858cf7da4d9faf8197b7a934a7c1, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:54.811668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d68e9d-e3e4-4eef-9f00-dc163b855fde admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:54.837711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1000/2009] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:55:54 2022] PUT /identity/v3/projects/3a67119e02b744e29dfe3ad1865798d2/users/9b8e858cf7da4d9faf8197b7a934a7c1/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:54.841937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:54.842337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:54.842812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:54.887235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] MFA Rules not processed for user `9b8e858cf7da4d9faf8197b7a934a7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:54.888643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:54.890783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1010/2010] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:54.904613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb4781-551f-4955-8903-62914d556908 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.944116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:54.972811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1001/2011] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:54.977653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb4781-551f-4955-8903-62914d556908 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:54.985246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:54.986684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:55.052998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:55.053953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:55.054308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:55.055010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:55.068347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed92f3-9ea7-4c19-898b-83dcd9a6956d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:55.074176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:55.075015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:55.075511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:55.076121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:55.095532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:55.097247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:55.111519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1002/2012] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:55.141812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:55.143496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:55.156695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1011/2013] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:54 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:55.170298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:55.187460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb24e6-2342-4e50-91ce-f3383b0573b3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:55.191660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:55.192933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:55.193220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:55.193491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:55.255311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:55.263930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d1b37-c803-418e-ba07-954463ca4457 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:55.267635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:55.268234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:55.268520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:55.268795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:55.281776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:55.284195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:55.289742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:55.299256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1003/2014] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:55:55.342174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:55.344093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7cc90-a8b2-4a22-9063-be74d136634e service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:55.355287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1012/2015] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:58.615821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:58.693649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85426-7b4c-4609-a874-2bab1099ac87 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:58.699093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:58.699789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:58.700071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:58.700371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] PATH_INFO: `/v3/users/45526c7557e64259afbe75b3043eb4bb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:58.701670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45526c7557e64259afbe75b3043eb4bb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:58.703681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:58.746384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:58.770984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45526c7557e64259afbe75b3043eb4bb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:58.771809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45526c7557e64259afbe75b3043eb4bb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:58.772747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45526c7557e64259afbe75b3043eb4bb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:58.807489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45526c7557e64259afbe75b3043eb4bb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:58.818427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45526c7557e64259afbe75b3043eb4bb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:58.836653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1004/2016] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:58 2022] DELETE /identity/v3/users/45526c7557e64259afbe75b3043eb4bb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:58.841053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:58.841700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:58.841700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:58.868383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:58.869657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:58.960266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1013/2017] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:55:58.972784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:58.974599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.051767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da4531-443f-4bfa-8e60-f286146b6928 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:59.058005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:59.059903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.059903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.059903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.079302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:59.084232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:59.099947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1005/2018] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:55:59.508633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:59.510723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.526267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:59.526838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb8fb-9b97-423a-b03e-6cf8511a4dad None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:59.527893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.550322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:59.550546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e763-c3a6-40f0-afa9-3052fa90975c service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:59.551663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.552232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.552963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] PATH_INFO: `/v3/users/381fb3ca62a747bba4820d5e69637956` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.554735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381fb3ca62a747bba4820d5e69637956)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:59.556969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:59.558239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:59.558875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.559949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.560666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] PATH_INFO: `/v3/projects/07f729325cb84b048432af82515783b2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.562829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f729325cb84b048432af82515783b2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:59.566368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:59.621877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381fb3ca62a747bba4820d5e69637956'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.633779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381fb3ca62a747bba4820d5e69637956'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.658131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f729325cb84b048432af82515783b2', 'request_id': 'req-52319c97-94c6-4786-b844-3440764d231e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.658761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381fb3ca62a747bba4820d5e69637956'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.667387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f729325cb84b048432af82515783b2', 'request_id': 'req-52319c97-94c6-4786-b844-3440764d231e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.670347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381fb3ca62a747bba4820d5e69637956'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.671401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fa3ceee3-c27d-4c45-b7fb-90951d40b844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381fb3ca62a747bba4820d5e69637956'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.680753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f729325cb84b048432af82515783b2', 'request_id': 'req-52319c97-94c6-4786-b844-3440764d231e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.692863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1014/2019] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:59 2022] DELETE /identity/v3/users/381fb3ca62a747bba4820d5e69637956 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:59.692863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.692863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.692863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.696827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] The token cache is being invalidate because project 07f729325cb84b048432af82515783b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:55:59.697285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.698470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.703262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1006/2020] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:55:59 2022] DELETE /identity/v3/projects/07f729325cb84b048432af82515783b2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:55:59.708718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:59.710807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.729334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:55:59.730584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.764953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:59.786212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:55:59.823613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52319c97-94c6-4786-b844-3440764d231e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:59.825252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1015/2021] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:55:59.826328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:59.827841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.830873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:59.831728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.832077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.833015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] PATH_INFO: `/v3/users/85f412229d6c41ca886d53984b1a932b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.834563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f412229d6c41ca886d53984b1a932b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:59.836977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:59.844361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42844c-0055-45a6-b2fb-5e52d8976452 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:55:59.852816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:55:59.854142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.854463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.854830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] PATH_INFO: `/v3/users/6292f50084be42cf999c00d6b66931c4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:55:59.856413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6292f50084be42cf999c00d6b66931c4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:55:59.858774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:55:59.893114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f412229d6c41ca886d53984b1a932b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.898406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f412229d6c41ca886d53984b1a932b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.899315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f412229d6c41ca886d53984b1a932b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.913805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f412229d6c41ca886d53984b1a932b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.918436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6292f50084be42cf999c00d6b66931c4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.923935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f412229d6c41ca886d53984b1a932b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.926286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6292f50084be42cf999c00d6b66931c4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.948258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6292f50084be42cf999c00d6b66931c4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.949628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1007/2022] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:59 2022] DELETE /identity/v3/users/85f412229d6c41ca886d53984b1a932b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:59.954788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:55:59.956323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:55:59.978052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6292f50084be42cf999c00d6b66931c4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.979035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f06f56db-bdcc-4016-8da1-a59c80aeee0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6292f50084be42cf999c00d6b66931c4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:55:59.987739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1016/2023] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:55:59 2022] DELETE /identity/v3/users/6292f50084be42cf999c00d6b66931c4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:55:59.992513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:55:59.993206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:55:59.993206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.026371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:00.027688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.062201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a29fdf-621f-45f5-820f-bd901a4f4035 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.110715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1017/2024] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:00.127733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.128358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.128637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.128910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.131014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.135742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.137217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.160094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259aea6-4994-4d47-bab1-f724a8307b46 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.162556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.164132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.165134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.165798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.166080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.166711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.177631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1008/2025] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:00.180251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.181755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.196020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1018/2026] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:00.466591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.470945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.488294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6354-34f0-4d97-b0b5-80fd75349f9e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.493958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.494810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.495565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.495963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] PATH_INFO: `/v3/projects/cef52abdf9a9485382412aabaa4a9091` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.497487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef52abdf9a9485382412aabaa4a9091)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.500202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.513125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:00.591883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef52abdf9a9485382412aabaa4a9091', 'request_id': 'req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.599007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef52abdf9a9485382412aabaa4a9091', 'request_id': 'req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.607183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef52abdf9a9485382412aabaa4a9091', 'request_id': 'req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.621256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.622109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] The token cache is being invalidate because project cef52abdf9a9485382412aabaa4a9091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:00.622407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.622782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.623075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.624772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1009/2027] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:00 2022] DELETE /identity/v3/projects/cef52abdf9a9485382412aabaa4a9091 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:00.628373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.631224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.640111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019ca-5766-41a2-aa59-018e219fec88 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.677989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:00.716245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa48ea-783c-4b3e-a29a-a6ac06cc2986 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.721324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.724936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.725227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.725507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] PATH_INFO: `/v3/users/9b8e858cf7da4d9faf8197b7a934a7c1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.727071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8e858cf7da4d9faf8197b7a934a7c1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.729114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.731819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.732865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.733031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.734022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] PATH_INFO: `/v3/projects/38d790fc1f674e5f983f2806d962bc70` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.734902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d790fc1f674e5f983f2806d962bc70)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.736803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.800810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8e858cf7da4d9faf8197b7a934a7c1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.803517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8e858cf7da4d9faf8197b7a934a7c1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.806087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8e858cf7da4d9faf8197b7a934a7c1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.825168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d790fc1f674e5f983f2806d962bc70', 'request_id': 'req-573ecb54-b09d-4573-b8c5-704ee848b259'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.826078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8e858cf7da4d9faf8197b7a934a7c1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.837400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8e858cf7da4d9faf8197b7a934a7c1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.840891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d790fc1f674e5f983f2806d962bc70', 'request_id': 'req-573ecb54-b09d-4573-b8c5-704ee848b259'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.849690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d790fc1f674e5f983f2806d962bc70', 'request_id': 'req-573ecb54-b09d-4573-b8c5-704ee848b259'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.853687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1010/2028] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:00 2022] DELETE /identity/v3/users/9b8e858cf7da4d9faf8197b7a934a7c1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:00.856613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.856613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.880136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:00.884751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] The token cache is being invalidate because project 38d790fc1f674e5f983f2806d962bc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:00.885061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.886049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:00.887891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1019/2029] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:00 2022] DELETE /identity/v3/projects/38d790fc1f674e5f983f2806d962bc70 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:00.891815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:00.893380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:00.953691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788325-b55e-45d1-894e-8d321fdf36d1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:00.959369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:00.961408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:00.961408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:00.961408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] PATH_INFO: `/v3/projects/0f06d27c2159414f88e9a803d8d0b86f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:00.967771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f06d27c2159414f88e9a803d8d0b86f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:00.967771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:00.996620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ecb54-b09d-4573-b8c5-704ee848b259 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.003620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.004870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.006045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.009141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] PATH_INFO: `/v3/users/2bae74f6c08e4344ab23ce7d271284cf` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.010711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bae74f6c08e4344ab23ce7d271284cf)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.013151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.035706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f06d27c2159414f88e9a803d8d0b86f', 'request_id': 'req-f95d1006-dd13-401a-b435-60383c9e8c80'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.050687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f06d27c2159414f88e9a803d8d0b86f', 'request_id': 'req-f95d1006-dd13-401a-b435-60383c9e8c80'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.064717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f06d27c2159414f88e9a803d8d0b86f', 'request_id': 'req-f95d1006-dd13-401a-b435-60383c9e8c80'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.085984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] The token cache is being invalidate because project 0f06d27c2159414f88e9a803d8d0b86f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:01.086289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.087189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.088017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bae74f6c08e4344ab23ce7d271284cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.089068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1011/2030] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:00 2022] DELETE /identity/v3/projects/0f06d27c2159414f88e9a803d8d0b86f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:01.093271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.097129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.100735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bae74f6c08e4344ab23ce7d271284cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.112269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:01.127631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bae74f6c08e4344ab23ce7d271284cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.140370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bae74f6c08e4344ab23ce7d271284cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.141082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bae74f6c08e4344ab23ce7d271284cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.149995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1020/2031] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:00 2022] DELETE /identity/v3/users/2bae74f6c08e4344ab23ce7d271284cf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:01.197049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d1006-dd13-401a-b435-60383c9e8c80 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.275079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.277361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.284418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:01.299919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.300809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.301381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.302042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] PATH_INFO: `/v3/users/78417e7f939a444ea46e418d47012246` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.303580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78417e7f939a444ea46e418d47012246)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.305632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.394339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78417e7f939a444ea46e418d47012246'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.395838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78417e7f939a444ea46e418d47012246'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.396714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78417e7f939a444ea46e418d47012246'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.416757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78417e7f939a444ea46e418d47012246'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.425079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ed715445-8f74-4984-9b60-f1acb9b72600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78417e7f939a444ea46e418d47012246'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.433280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d13e6-fc96-49ae-acda-a05621fbdd3c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.440196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.440862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.441141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.441413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] PATH_INFO: `/v3/projects/3a67119e02b744e29dfe3ad1865798d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.443662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a67119e02b744e29dfe3ad1865798d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.445633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.447269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1012/2032] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:01 2022] DELETE /identity/v3/users/78417e7f939a444ea46e418d47012246 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:01.529478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a67119e02b744e29dfe3ad1865798d2', 'request_id': 'req-1cfea5ad-14c1-493c-bd60-e151394bbc91'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.540122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a67119e02b744e29dfe3ad1865798d2', 'request_id': 'req-1cfea5ad-14c1-493c-bd60-e151394bbc91'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.548561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a67119e02b744e29dfe3ad1865798d2', 'request_id': 'req-1cfea5ad-14c1-493c-bd60-e151394bbc91'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.566723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] The token cache is being invalidate because project 3a67119e02b744e29dfe3ad1865798d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:01.567593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.568316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1cfea5ad-14c1-493c-bd60-e151394bbc91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.569966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1021/2033] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:01 2022] DELETE /identity/v3/projects/3a67119e02b744e29dfe3ad1865798d2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:01.576239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.576594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.576944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.601379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:01.602430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.670724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1013/2034] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:01.681167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.681555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.681944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.687370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.693535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.710709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978dbf1-5cf1-4a95-a919-0770a916ef31 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.716734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.720085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.720377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.720653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] PATH_INFO: `/v3/projects/82368c4e1901456f8e0da482a2869454` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.722306 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82368c4e1901456f8e0da482a2869454)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.723962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.739240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:01.740761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.759291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1022/2035] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:01.767042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.777981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec66d4-243a-4f74-9d7e-bec2f597fc44 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.784056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.784412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.784629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.784891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.785626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:01.785971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.788565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.798275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82368c4e1901456f8e0da482a2869454', 'request_id': 'req-39db2562-b89a-4ec3-87b3-4d02e787133d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.806347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1023/2036] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:01.807348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.809546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82368c4e1901456f8e0da482a2869454', 'request_id': 'req-39db2562-b89a-4ec3-87b3-4d02e787133d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.821750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fc0d3-e3ea-4dd4-8564-9160905c46a0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.822602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82368c4e1901456f8e0da482a2869454', 'request_id': 'req-39db2562-b89a-4ec3-87b3-4d02e787133d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.827116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:01.838823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] The token cache is being invalidate because project 82368c4e1901456f8e0da482a2869454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:01.839074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.840070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.842275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1014/2037] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:01 2022] DELETE /identity/v3/projects/82368c4e1901456f8e0da482a2869454 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:01.843070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:01.844435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.886183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:01.898403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.899136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.899586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.900005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.900885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.902815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf70d4-c662-49c9-834f-24994455c162 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.915765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2562-b89a-4ec3-87b3-4d02e787133d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:01.921322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:01.922100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.922360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.922655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] PATH_INFO: `/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.923898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd78f6d10d347d1ac26aee14399c1f0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:01.925933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:01.933205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1024/2038] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:01 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:56:01.938156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815b2fc-f2e3-469e-b60c-dc78c4cf2760 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:01.938446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815b2fc-f2e3-469e-b60c-dc78c4cf2760 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:01.938728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815b2fc-f2e3-469e-b60c-dc78c4cf2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:01.959975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9815b2fc-f2e3-469e-b60c-dc78c4cf2760 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:01.961199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9815b2fc-f2e3-469e-b60c-dc78c4cf2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:01.975933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd78f6d10d347d1ac26aee14399c1f0', 'request_id': 'req-62f89e1c-4fe8-41c3-b54e-6c86663dec94'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.981977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd78f6d10d347d1ac26aee14399c1f0', 'request_id': 'req-62f89e1c-4fe8-41c3-b54e-6c86663dec94'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:01.990507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd78f6d10d347d1ac26aee14399c1f0', 'request_id': 'req-62f89e1c-4fe8-41c3-b54e-6c86663dec94'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:02.002297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] The token cache is being invalidate because project 1cd78f6d10d347d1ac26aee14399c1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:02.002583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:02.003244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:02.005677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1015/2039] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:01 2022] DELETE /identity/v3/projects/1cd78f6d10d347d1ac26aee14399c1f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:02.006490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.008010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.024713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1025/2040] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.026014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177b47-3ea1-4474-aae1-739b1d5a52d6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.026299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177b47-3ea1-4474-aae1-739b1d5a52d6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.026572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177b47-3ea1-4474-aae1-739b1d5a52d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.046645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6b177b47-3ea1-4474-aae1-739b1d5a52d6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:02.047846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6b177b47-3ea1-4474-aae1-739b1d5a52d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.056648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1026/2041] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.057675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201dc0-a4e6-46c2-9690-18fda22ddc21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.057958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201dc0-a4e6-46c2-9690-18fda22ddc21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.058229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201dc0-a4e6-46c2-9690-18fda22ddc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.077945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2f201dc0-a4e6-46c2-9690-18fda22ddc21 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:02.079169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f201dc0-a4e6-46c2-9690-18fda22ddc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.079705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f89e1c-4fe8-41c3-b54e-6c86663dec94 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.084829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.085444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.085750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.086024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.086927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.087288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1027/2042] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.088471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.088760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.088872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.089037 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.110179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:02.111391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.113277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1016/2043] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.114453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.123286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1028/2044] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.123995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.124738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e79f-ccb5-4b21-afba-7afce25861a9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.130479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.131443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.131443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.131966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.132281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:02.132590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.133644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f389f3-f446-4bfb-9ed8-f1530cb425eb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.134248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.139179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.139877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.140159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.140426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.141209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.142880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.147404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1017/2045] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:02.151732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.153885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1029/2046] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.154565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.157095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:02.163207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d42ef3-4797-42f3-b8e5-a96b0ef4eb5f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.165325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeefd-37c2-44fa-9f0e-9aa98a96b4e3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.169523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.170542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.170657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.171212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.171336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.171493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.171599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.171771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.172189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:02.172390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.172686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.174336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.174612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.187120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1018/2047] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:02.187965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.198847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd64c0b-a27e-4f75-812c-18213731ea62 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.204288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.204896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.205204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.205476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.206156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.206821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1030/2048] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.208229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.208393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.220601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1019/2049] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.221451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.238719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd1d64-a367-40a0-8693-53795edbf432 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.280247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2564f4-684c-4f12-9851-ead800cbfd2a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.286126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.286755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.287305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.287305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.287934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.291624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.320103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.320835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.321219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.321626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.322379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.324643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.333945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1031/2050] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.335342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.349367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:02.367713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1020/2051] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.374828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.388266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f0b1-d1e1-461b-b349-3dfdbed44388 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.430301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3520c-5019-48f0-88aa-4ede3b1a904e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.435561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.436240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.436528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.436851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] PATH_INFO: `/v3/projects/ac103ae137fe420f92e83489ea29f7a4/users/37cfc71bc5cb4a278128a22f48794179/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.452964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac103ae137fe420f92e83489ea29f7a4, user_id=37cfc71bc5cb4a278128a22f48794179, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.455907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.464643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.465299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.465579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.465883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.466523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.468470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128606e-0f91-4bf1-933a-023adfcf32dd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.471522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1032/2052] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:02 2022] PUT /identity/v3/projects/ac103ae137fe420f92e83489ea29f7a4/users/37cfc71bc5cb4a278128a22f48794179/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:02.472670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.503419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1021/2053] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.503419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.503419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.503419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.533957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] MFA Rules not processed for user `37cfc71bc5cb4a278128a22f48794179`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:02.535086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.560884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076eee2-a15b-498a-b217-1eebb3cf2018 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.565736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.567613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.567902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.568180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.569717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.571345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.582296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1033/2054] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.583148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.594475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:02.595935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c575cda-f708-4741-96fc-55255df76475 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.603078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.603954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.604395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.604817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.605634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.608454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.618447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1022/2055] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.619592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.624446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1034/2056] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.624446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.636379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f14d6b-e21d-49e0-b85f-8391d55b0072 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.636791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba9ee-7a1c-44ba-ae7f-d4d44170d2a2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.641423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.641675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.642096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.642362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.642498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.642637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.642844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.643185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.643292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.643919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.644937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.646018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.660612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1023/2057] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.660612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.672804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9136c-7dfc-46c3-8ef9-91b2981efb7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.678992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.679690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.680036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.680384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.681158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.683070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.687438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1035/2058] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.687438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.705270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1024/2059] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:02.706167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.725166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91febe5d-a3ef-4c08-be11-15d4b46b2980 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.759266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:02.765302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67e759-34af-4dbf-ae18-4363d022603c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.771909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.772741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.773132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.773501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] PATH_INFO: `/v3/projects/f3d3f35aa36c4f4884dc5ae71df254b8/users/62e902939e1647febeac0ba0d27eaa47/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.796612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.797147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.797901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.797901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.798120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d3f35aa36c4f4884dc5ae71df254b8, user_id=62e902939e1647febeac0ba0d27eaa47, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.798356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.800454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565fd2-a29e-4b4a-9673-36e3f52a83ab admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.801734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.817931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1036/2060] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:02 2022] PUT /identity/v3/projects/f3d3f35aa36c4f4884dc5ae71df254b8/users/62e902939e1647febeac0ba0d27eaa47/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:02.819230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.836641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1025/2061] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:02.838169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83916e9b-845c-4430-972b-c80c84653423 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.838656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83916e9b-845c-4430-972b-c80c84653423 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.839101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83916e9b-845c-4430-972b-c80c84653423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.870586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-83916e9b-845c-4430-972b-c80c84653423 None None] MFA Rules not processed for user `62e902939e1647febeac0ba0d27eaa47`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:02.872039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83916e9b-845c-4430-972b-c80c84653423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:02.899587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0ef4a-3737-4b95-8982-af624006c7b1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:02.906038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:02.907335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:02.907585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:02.907963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] PATH_INFO: `/v3/projects/54c62e03544e45cfa894b03c62232c48/users/693336077c734d9d948e06d6420db10a/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:02.928542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c62e03544e45cfa894b03c62232c48, user_id=693336077c734d9d948e06d6420db10a, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:02.931441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:02.947631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1026/2062] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:02.947631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83916e9b-845c-4430-972b-c80c84653423 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.959791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1037/2063] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:02 2022] PUT /identity/v3/projects/54c62e03544e45cfa894b03c62232c48/users/693336077c734d9d948e06d6420db10a/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:02.959791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:02.966398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83916e9b-845c-4430-972b-c80c84653423 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.016584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:03.033745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.034472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.034750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.035048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.036323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.038010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505559c-798c-464b-8f42-a4f3dee9a528 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.038465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2957-8e91-4129-8e42-26c67b7f1f71 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.044301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.044616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.044985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.045266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.046879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.048803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.054657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1027/2064] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:03.054657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.054657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.054657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.084710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1038/2065] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:02 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:03.085876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] MFA Rules not processed for user `693336077c734d9d948e06d6420db10a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:03.086803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.087118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:03.142296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:03.161366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21f64-1bf8-4c53-9b2c-147599316c3c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.167163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.167957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.168302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.168639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] PATH_INFO: `/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600/users/efeb58a8d627476aabbc612cc175af22/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.177400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1028/2066] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:03.177400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.185624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cccd2b7f6b49e595dc5096d8bf1600, user_id=efeb58a8d627476aabbc612cc175af22, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.188056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.206905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccc8ad-fda2-47f5-b337-7cd8aa0f2853 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.208759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1039/2067] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:03 2022] PUT /identity/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600/users/efeb58a8d627476aabbc612cc175af22/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:03.209663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.212959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:03.275804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.276536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.276890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.277332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.278211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.279353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d5bd-6ab7-46b5-88fe-c013d917fe38 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.280246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.283285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.283978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.284332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.284675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.308422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1029/2068] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:03 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:03.310450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.325089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc695e2-ccd6-467a-9657-c13fad92c82f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.333394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.334221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.336047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.336047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.336747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.339325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.352072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1030/2069] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:03.353125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.358075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.360099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.366524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3da46a-5749-4b45-99c5-9cad969fe0ce admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.376024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1040/2070] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:03.376851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.382465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.384136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.384136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.384536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.386878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.388894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.389945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b020508-ad84-4aa9-bed1-6e65381e4a25 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.398300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.398985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.399279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.399551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] PATH_INFO: `/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600/users/efeb58a8d627476aabbc612cc175af22/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.401615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cccd2b7f6b49e595dc5096d8bf1600, user_id=efeb58a8d627476aabbc612cc175af22, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.403760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1031/2071] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:03.406759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59d269-36ba-4ef4-8452-1b5ed2a7cd60 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.408229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.435165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a349e-1afc-42cc-adf6-e1ced8709562 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.439270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1041/2072] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:03 2022] PUT /identity/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600/users/efeb58a8d627476aabbc612cc175af22/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:03.451826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.452140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.452429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.490955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] MFA Rules not processed for user `efeb58a8d627476aabbc612cc175af22`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:03.493068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:03.498272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:03.529736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.530783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.531185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.531561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] PATH_INFO: `/v3/projects/96f749e30e314ea8875a4ac786e0a713/users/a5caab6fc4874e7b95516572ada60f79/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.559272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f749e30e314ea8875a4ac786e0a713, user_id=a5caab6fc4874e7b95516572ada60f79, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.563067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.578863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:03.592288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1032/2073] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:03 2022] PUT /identity/v3/projects/96f749e30e314ea8875a4ac786e0a713/users/a5caab6fc4874e7b95516572ada60f79/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:03.595971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.599983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1042/2074] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:03.614989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.617143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:03.674243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3ebea-99f6-44d2-965a-99989827a8f3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.687228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.693627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.693627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.693627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.693627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.694027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850001b-0b33-47fc-86e4-b181c8074034 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.697270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.738047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1033/2075] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:03.742957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:03.753048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cb8c9-f49e-4e1a-a416-65b2056cde2e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:03.768755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.769673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.770483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.775426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] PATH_INFO: `/v3/projects/96f749e30e314ea8875a4ac786e0a713/users/a5caab6fc4874e7b95516572ada60f79/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.777293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f749e30e314ea8875a4ac786e0a713, user_id=a5caab6fc4874e7b95516572ada60f79, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.783589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a19e-5c1e-4eae-9aa9-a11dfd4ee0cc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.806598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:03.807231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.807514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.807791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.810636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1034/2076] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:03 2022] PUT /identity/v3/projects/96f749e30e314ea8875a4ac786e0a713/users/a5caab6fc4874e7b95516572ada60f79/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:03.814716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:03.815522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:03.815522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:03.859204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] MFA Rules not processed for user `a5caab6fc4874e7b95516572ada60f79`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:03.860457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:03.904700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:03.906212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:03.930069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1043/2077] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:03.979904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1035/2078] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:03.998208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:04.002078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab586-6cc4-4d31-a305-545e7b6f582c service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:04.009531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:04.059824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:04.076591 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:04.077554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:04.077936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:04.078425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:04.089194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790183e9-b4e5-4887-8388-7596c084af8b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:04.089951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:04.091450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:04.094115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:04.095326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:04.095326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:04.095569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:04.105274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1044/2079] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:04.182460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:04.183918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:04.194830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1036/2080] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:56:04.597749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:04.610065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:04.711750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee896-8a7a-4ab5-b870-a00833ee8c2f service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:04.726283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:04.726949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:04.727231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:04.727508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] PATH_INFO: `/v3/users/37cfc71bc5cb4a278128a22f48794179` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:04.728668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37cfc71bc5cb4a278128a22f48794179)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:04.730644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:04.803262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:04.822534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124875f-bf07-4737-a03a-95ae6ff374ff service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:04.830571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:04.831229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:04.831509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:04.831787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] PATH_INFO: `/v3/users/62e902939e1647febeac0ba0d27eaa47` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:04.833534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e902939e1647febeac0ba0d27eaa47)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:04.834147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37cfc71bc5cb4a278128a22f48794179'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.835676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:04.853171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37cfc71bc5cb4a278128a22f48794179'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.878395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37cfc71bc5cb4a278128a22f48794179'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.906244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37cfc71bc5cb4a278128a22f48794179'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.907542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d6e6091e-18fb-4c05-9c8d-62ed8fd649b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37cfc71bc5cb4a278128a22f48794179'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.917646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1045/2081] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:04 2022] DELETE /identity/v3/users/37cfc71bc5cb4a278128a22f48794179 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:04.923830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:04.924309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:04.925203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:04.934292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e902939e1647febeac0ba0d27eaa47'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.937000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e902939e1647febeac0ba0d27eaa47'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.939694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e902939e1647febeac0ba0d27eaa47'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:04.984995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:04.989645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:04.992409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e902939e1647febeac0ba0d27eaa47'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:05.017901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-41f564a6-eab1-4259-85af-0fbb6b482a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e902939e1647febeac0ba0d27eaa47'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:05.057811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1037/2082] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:04 2022] DELETE /identity/v3/users/62e902939e1647febeac0ba0d27eaa47 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:05.063304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:05.063807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:05.064303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:05.140970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:05.146159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:05.224852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1046/2083] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:05.247147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:05.251197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:05.275718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:05.296757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1038/2084] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:05.308412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:05.312102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:05.361663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:05.417570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa833-8a37-4367-8f5e-8c14850452ca None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:05.422079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef953d-1ee3-4bc7-a241-24ffdd328423 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:05.422317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:05.423055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:05.423335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:05.423603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:05.431806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:05.432625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:05.433193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:05.433365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:05.436801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:05.438979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:05.448531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:05.452145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:05.452824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1039/2085] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:05.466867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1047/2086] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:05.952097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:05.953838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:05.977088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f971d-8f43-44c3-8feb-3c12f3e68ca8 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:05.986470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:05.988209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:05.988845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:05.989276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] PATH_INFO: `/v3/projects/ac103ae137fe420f92e83489ea29f7a4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:05.990705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac103ae137fe420f92e83489ea29f7a4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:06.002426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:06.071468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac103ae137fe420f92e83489ea29f7a4', 'request_id': 'req-a5322cd9-ae89-463c-8093-20e3d41b5a8f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.078543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac103ae137fe420f92e83489ea29f7a4', 'request_id': 'req-a5322cd9-ae89-463c-8093-20e3d41b5a8f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.092577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac103ae137fe420f92e83489ea29f7a4', 'request_id': 'req-a5322cd9-ae89-463c-8093-20e3d41b5a8f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.111202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] The token cache is being invalidate because project ac103ae137fe420f92e83489ea29f7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:06.111202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.112002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.113489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1040/2087] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:05 2022] DELETE /identity/v3/projects/ac103ae137fe420f92e83489ea29f7a4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:06.118450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:06.164555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2565c9-6bd8-41c6-af92-41957e1126d5 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:06.242082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:06.244181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:06.300596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:06.301795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:06.302480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:06.302971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] PATH_INFO: `/v3/users/efeb58a8d627476aabbc612cc175af22` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:06.304686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efeb58a8d627476aabbc612cc175af22)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:06.307319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:06.410862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322cd9-ae89-463c-8093-20e3d41b5a8f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:06.419821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:06.420768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:06.421233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:06.421630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] PATH_INFO: `/v3/projects/f3d3f35aa36c4f4884dc5ae71df254b8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:06.423093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d3f35aa36c4f4884dc5ae71df254b8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:06.425193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:06.432658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efeb58a8d627476aabbc612cc175af22'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.441248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efeb58a8d627476aabbc612cc175af22'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.457674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efeb58a8d627476aabbc612cc175af22'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.474074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:06.477494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efeb58a8d627476aabbc612cc175af22'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.478469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efeb58a8d627476aabbc612cc175af22'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.493642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1048/2088] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:06 2022] DELETE /identity/v3/users/efeb58a8d627476aabbc612cc175af22 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:06.493642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:06.497938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:06.534554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d3f35aa36c4f4884dc5ae71df254b8', 'request_id': 'req-07796d68-034a-4c65-8447-9b07486e07d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.543720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d3f35aa36c4f4884dc5ae71df254b8', 'request_id': 'req-07796d68-034a-4c65-8447-9b07486e07d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.562089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d3f35aa36c4f4884dc5ae71df254b8', 'request_id': 'req-07796d68-034a-4c65-8447-9b07486e07d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.586408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] The token cache is being invalidate because project f3d3f35aa36c4f4884dc5ae71df254b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:06.586707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.588395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.590436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1041/2089] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:06 2022] DELETE /identity/v3/projects/f3d3f35aa36c4f4884dc5ae71df254b8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:06.594716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:06.597068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:06.626432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:06.658308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8742d0e-277a-4093-8517-11a2337f23c7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:06.798333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:06.798997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:06.799281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:06.799701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] PATH_INFO: `/v3/users/bce16ba58a534dff8f132aec1e3f8999` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:06.801269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce16ba58a534dff8f132aec1e3f8999)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:06.805577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:06.824600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796d68-034a-4c65-8447-9b07486e07d8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:06.845028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:06.845862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:06.847403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:06.847693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] PATH_INFO: `/v3/users/a5caab6fc4874e7b95516572ada60f79` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:06.850644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5caab6fc4874e7b95516572ada60f79)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:06.855724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:06.938973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce16ba58a534dff8f132aec1e3f8999'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.967228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce16ba58a534dff8f132aec1e3f8999'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:06.991710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce16ba58a534dff8f132aec1e3f8999'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.011822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce16ba58a534dff8f132aec1e3f8999'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.012043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5caab6fc4874e7b95516572ada60f79'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.012687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c89dc123-e61e-4e6d-884b-71f0092c85b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce16ba58a534dff8f132aec1e3f8999'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.013313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5caab6fc4874e7b95516572ada60f79'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.014931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5caab6fc4874e7b95516572ada60f79'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.021633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1049/2090] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:06 2022] DELETE /identity/v3/users/bce16ba58a534dff8f132aec1e3f8999 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:07.026918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.027210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.027488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.052018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5caab6fc4874e7b95516572ada60f79'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.058106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:07.059354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.059509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5caab6fc4874e7b95516572ada60f79'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.074664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1042/2091] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:06 2022] DELETE /identity/v3/users/a5caab6fc4874e7b95516572ada60f79 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:07.126797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1050/2092] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:07.142440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.143999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.181713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:07.212082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7361aa5-959d-4378-8adf-e56e5f2b23d2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.219803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.220640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.221916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.222333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.229341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.230872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.247399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829f209-1f2d-4af6-8561-7208bf9fb872 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.266069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:07.304181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.306102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36390b-711d-4b9b-bd42-27d62ae79f10 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:07.320523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1043/2093] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:56:07.338776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.339632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.339981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.340298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] PATH_INFO: `/v3/users/693336077c734d9d948e06d6420db10a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.341768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=693336077c734d9d948e06d6420db10a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.343970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:07.407414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '693336077c734d9d948e06d6420db10a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.417438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '693336077c734d9d948e06d6420db10a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.435812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '693336077c734d9d948e06d6420db10a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.449874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '693336077c734d9d948e06d6420db10a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.450635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '693336077c734d9d948e06d6420db10a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.467847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1051/2094] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:07 2022] DELETE /identity/v3/users/693336077c734d9d948e06d6420db10a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:07.474059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1de63-5070-4cbc-8c31-a1adf498380c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.474354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1de63-5070-4cbc-8c31-a1adf498380c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.474680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1de63-5070-4cbc-8c31-a1adf498380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.481502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.486837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.510221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-13e1de63-5070-4cbc-8c31-a1adf498380c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:07.512240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-13e1de63-5070-4cbc-8c31-a1adf498380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.654911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1044/2095] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:07.654911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.654911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.654911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.654911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1045/2096] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:56:07.664568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.670437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.681221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31197bb1-97cf-4422-84b0-a199f5a61943 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.686165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.686819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.687927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.688045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] PATH_INFO: `/v3/projects/96f749e30e314ea8875a4ac786e0a713` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.689261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f749e30e314ea8875a4ac786e0a713)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.691250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:07.758307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:07.764109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116ee53-f226-47fd-9e19-045e0a14fc1b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.798400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f749e30e314ea8875a4ac786e0a713', 'request_id': 'req-ba2d0871-6d82-424a-806d-d80521223e41'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.810379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:07.811182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f749e30e314ea8875a4ac786e0a713', 'request_id': 'req-ba2d0871-6d82-424a-806d-d80521223e41'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.819444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f749e30e314ea8875a4ac786e0a713', 'request_id': 'req-ba2d0871-6d82-424a-806d-d80521223e41'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.838923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] The token cache is being invalidate because project 96f749e30e314ea8875a4ac786e0a713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:07.839229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.839944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba2d0871-6d82-424a-806d-d80521223e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:07.841673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.841811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1052/2097] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:07 2022] DELETE /identity/v3/projects/96f749e30e314ea8875a4ac786e0a713 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:07.842933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.843269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.843642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.847171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.848526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.848837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.851145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.867560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:07.868995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.922731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.924125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:07.933986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1046/2098] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:56:07.934931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.936313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.943088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1053/2099] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:56:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:07.944315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:07.946807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:07.949858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6119-9f87-4171-832a-87d65c925067 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.954702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.955347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.955743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.956052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] PATH_INFO: `/v3/projects/7d8c56658d75461fa91c8a393019aee5` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.957035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8c56658d75461fa91c8a393019aee5)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.960160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:07.963204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb536a-e400-4ef6-8060-fd19f4822a59 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:07.968232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:07.968969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:07.969474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:07.969791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] PATH_INFO: `/v3/users/1bbaf0feaf294fc79f8f7d8dbcf58851` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:07.971050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbaf0feaf294fc79f8f7d8dbcf58851)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:07.973100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.012816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8c56658d75461fa91c8a393019aee5', 'request_id': 'req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.021010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8c56658d75461fa91c8a393019aee5', 'request_id': 'req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.028265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbaf0feaf294fc79f8f7d8dbcf58851'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.031655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8c56658d75461fa91c8a393019aee5', 'request_id': 'req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.036175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbaf0feaf294fc79f8f7d8dbcf58851'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.048307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] The token cache is being invalidate because project 7d8c56658d75461fa91c8a393019aee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:08.048904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.049782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e8e5ef8-1d63-4e46-a038-e543727d8e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.050736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbaf0feaf294fc79f8f7d8dbcf58851'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.051354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1047/2100] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:07 2022] DELETE /identity/v3/projects/7d8c56658d75461fa91c8a393019aee5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:08.052697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.052986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.053275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.060284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbaf0feaf294fc79f8f7d8dbcf58851'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.061163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbaf0feaf294fc79f8f7d8dbcf58851'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.073583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1054/2101] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:07 2022] DELETE /identity/v3/users/1bbaf0feaf294fc79f8f7d8dbcf58851 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:08.073583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.073583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.085219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.088592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.118583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:08.158004 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8a7f1-1b97-4310-8592-6ac2d90f4260 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.164215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.164852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.165165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.165653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] PATH_INFO: `/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.167546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4cccd2b7f6b49e595dc5096d8bf1600)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.169460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.173015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1048/2102] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.173912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.175242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.190213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:08.231497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4cccd2b7f6b49e595dc5096d8bf1600', 'request_id': 'req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.243789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4cccd2b7f6b49e595dc5096d8bf1600', 'request_id': 'req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.262265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4cccd2b7f6b49e595dc5096d8bf1600', 'request_id': 'req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.278557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea95754-1c9c-4fbf-9736-3d4a43b20a7e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.280827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] The token cache is being invalidate because project e4cccd2b7f6b49e595dc5096d8bf1600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:08.280827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.281400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32588a07-3c1a-40f3-9c98-43eb48bdc2d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.283631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1055/2103] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:08 2022] DELETE /identity/v3/projects/e4cccd2b7f6b49e595dc5096d8bf1600 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:08.284802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fc9a-2b20-48b6-a4da-65ff601540c0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.285091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fc9a-2b20-48b6-a4da-65ff601540c0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.285906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fc9a-2b20-48b6-a4da-65ff601540c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.313002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2ad8fc9a-2b20-48b6-a4da-65ff601540c0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.314266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2ad8fc9a-2b20-48b6-a4da-65ff601540c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.376204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.376819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.377100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.377383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.379259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.387311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1056/2104] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.388647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a9b1-2ad3-4776-a77a-b901b96a0a17 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.388998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a9b1-2ad3-4776-a77a-b901b96a0a17 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.389336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a9b1-2ad3-4776-a77a-b901b96a0a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.397437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.399133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.408617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a4f3a9b1-2ad3-4776-a77a-b901b96a0a17 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.409253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1049/2105] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:08.410176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.410984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4f3a9b1-2ad3-4776-a77a-b901b96a0a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.411616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.419496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1057/2106] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.424164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.424398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.424682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.429538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe9548-c1be-41c6-9f13-2116ed30e4d0 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.435337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.436561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.436561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.436739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] PATH_INFO: `/v3/projects/54c62e03544e45cfa894b03c62232c48` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.437917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c62e03544e45cfa894b03c62232c48)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.439903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.447832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.449095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.456979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1058/2107] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.458470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.459687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.471951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ad1e-47c7-48ba-8584-8f915b597995 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.479067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.479733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.480012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.480290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] PATH_INFO: `/v3/projects/6604f675c98a470db3568612140f7e8e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.481428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6604f675c98a470db3568612140f7e8e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.483389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.496768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c62e03544e45cfa894b03c62232c48', 'request_id': 'req-0a8f8b10-9bde-4308-b266-f3ad713590ef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.502203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c62e03544e45cfa894b03c62232c48', 'request_id': 'req-0a8f8b10-9bde-4308-b266-f3ad713590ef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.511079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c62e03544e45cfa894b03c62232c48', 'request_id': 'req-0a8f8b10-9bde-4308-b266-f3ad713590ef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.524524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] The token cache is being invalidate because project 54c62e03544e45cfa894b03c62232c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:08.524979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.525679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0a8f8b10-9bde-4308-b266-f3ad713590ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.527232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1050/2108] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:08 2022] DELETE /identity/v3/projects/54c62e03544e45cfa894b03c62232c48 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:08.528408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd468-64b7-4066-bebd-806606898c26 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.528710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd468-64b7-4066-bebd-806606898c26 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.528964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fd468-64b7-4066-bebd-806606898c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.534198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:08.542982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6604f675c98a470db3568612140f7e8e', 'request_id': 'req-0cc98309-0a3d-4288-9141-2662bb630d50'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.552694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b50fd468-64b7-4066-bebd-806606898c26 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.554069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6604f675c98a470db3568612140f7e8e', 'request_id': 'req-0cc98309-0a3d-4288-9141-2662bb630d50'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.554253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b50fd468-64b7-4066-bebd-806606898c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.563295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6604f675c98a470db3568612140f7e8e', 'request_id': 'req-0cc98309-0a3d-4288-9141-2662bb630d50'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.575491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] The token cache is being invalidate because project 6604f675c98a470db3568612140f7e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:08.575794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.576527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:08.578046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:08.578353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1059/2109] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:08 2022] DELETE /identity/v3/projects/6604f675c98a470db3568612140f7e8e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:08.579132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.580464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.633634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1051/2110] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.635190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c4f6-68a3-40b3-a996-8c70d2a2b92f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.635543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c4f6-68a3-40b3-a996-8c70d2a2b92f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.635880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c4f6-68a3-40b3-a996-8c70d2a2b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.655073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e8e9c4f6-68a3-40b3-a996-8c70d2a2b92f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.656273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e8e9c4f6-68a3-40b3-a996-8c70d2a2b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.660205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98309-0a3d-4288-9141-2662bb630d50 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.664706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1052/2111] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.665803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.666063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.666599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.666876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.667333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.667772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.668169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.668966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:08.669306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.670950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456f031-2d9b-46d5-89ae-120d82123bdc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.685590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1060/2112] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:08.686612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.686897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.687167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.690704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.692524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.701643 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1053/2113] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.702675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.709158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.710403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.713926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4431a3-7f01-405d-816b-d2c0387b1a4a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.719454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.721728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1061/2114] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.721728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.722390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.722590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.723410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.724085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:08.724396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.727632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.731100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fc0aa-3232-4c19-964f-0201d10a0f58 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.735669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.736263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.736540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.736809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.737449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:08.737778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.739989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e20a2a-adc8-43d9-916b-bf9fe8119b67 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.743564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1054/2115] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:08.743564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.750731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ee33d-879b-4101-b6a5-552edb8b5621 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.752747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1062/2116] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:08.754017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf350a-39a7-4586-a0af-bc0a5ee7a6e7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.754418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf350a-39a7-4586-a0af-bc0a5ee7a6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.754748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf350a-39a7-4586-a0af-bc0a5ee7a6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.755534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.756512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.756512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.756690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.757278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.759219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.774247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f7bf350a-39a7-4586-a0af-bc0a5ee7a6e7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.775454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f7bf350a-39a7-4586-a0af-bc0a5ee7a6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.783644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1063/2117] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.784819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.785112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.785923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.790746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1055/2118] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:08.791904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.812575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:08.813998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:08.867498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b4220-dbb4-44ad-9613-af2a340fc72b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.875867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.876491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.876767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.877117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.877706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.879750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.894142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1064/2119] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:08.895784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.909967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf59d4-7e2f-4570-92bc-1f98f504acc3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:08.919918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:08.920402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1056/2120] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:08.920537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:08.920817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:08.921117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:08.921221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.921905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:08.923871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:08.957691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1065/2121] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/projects => generated 361 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:08.958897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:08.984645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:08.998921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4836096-ae72-400c-aa01-648d52457671 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.011711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:09.046579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc13c06-9a1f-4aca-abfa-a2ec70af3f0c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.052891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.053526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.053847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.054124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.054784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.056745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.063976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.064683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.064868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.065149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.065854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:09.066165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.067814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.081933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1066/2122] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:08 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.081933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.082723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1057/2123] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:09.082723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.091851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5b16f-a243-4d08-a8de-d403efdefcea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.093245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa4394-b930-45dc-8741-dafc64c0a0de admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.096699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.097295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.097578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.097960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.099067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.099229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.099903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.100176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.100504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.100996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.101099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:09.101367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.102998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.116798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1067/2124] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:09.117633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.126114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1058/2125] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.127292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6adb-42b2-4a40-8cc7-f5d8484ae795 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.127427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.132965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.133587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.133901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.134194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.134838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.136747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.137342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cab12-40dd-42b6-8d74-3ee61c47c0ac admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.142026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.142618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.142896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.143166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.143799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.146022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.158282 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1059/2126] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.158912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.165332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1068/2127] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/users => generated 281 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.166130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.173793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d099f2-78bf-4815-bdea-8b8caae9f377 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.179503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3f48-d2f0-45af-b1ed-b1086779c20f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.180705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.181476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.181782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.182068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.182708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.184529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.184809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.185145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.185440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.185749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.186385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.188353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.214201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1060/2128] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.214969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.219949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1069/2129] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.220837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.283281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69c880-b603-4719-84bc-e56da6c0406d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.285450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c819ff8-248a-4008-b8dd-415fa9f7dc83 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.288752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.289581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.289891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.290170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.290790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.290930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.291654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.291937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.292338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.292701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.292992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.294625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.303092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1061/2130] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.303918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.306963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1070/2131] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.306963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.313960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827709-93c8-448a-a298-de6a57025a46 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.316233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccb184-751c-4594-bf5f-73be0935ff81 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.318746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.319345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.319626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.319899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.320538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.321055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.321682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.321967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.322184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.322277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.322854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.324744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.332993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1062/2132] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.336349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.344096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e1a26-c824-4640-9673-5f5b1040aab5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.349661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.350294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1071/2133] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.350878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.351366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.351484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.352062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.352843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.355262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.361469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4b6a5-ea6a-45ab-910b-61d81d39ceb5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.367907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.368609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.368964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.369240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.370117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.371817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.377897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1063/2134] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:09.378710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.385674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1072/2135] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.386205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.388993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea034e9-3b42-440f-a03a-eb43f5dba364 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.394193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.394880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.395164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.395472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] PATH_INFO: `/v3/projects/873d8d780a644e38b303cc76e17f3231/users/8b7c32f5b53f41368ba22c95ea7b2011/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.396171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ade981-dfc9-47fc-8735-d21f7cf639da admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.401456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.402077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.402356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.402629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.403271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.404848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.410581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873d8d780a644e38b303cc76e17f3231, user_id=8b7c32f5b53f41368ba22c95ea7b2011, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.413287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.415620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1073/2136] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.417771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.422706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:09.429279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1268f-e2dd-4cde-86e9-a3d32372261b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.430701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1064/2137] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:09 2022] PUT /identity/v3/projects/873d8d780a644e38b303cc76e17f3231/users/8b7c32f5b53f41368ba22c95ea7b2011/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:09.432103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.446274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:09.497044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498079e-21ad-4b9f-9009-50e593c39690 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.501156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.501637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.501822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.502119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.502257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.502430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.502526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.502787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.503086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.503430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.505352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.505553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.515902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1074/2138] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.516834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.519362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1065/2139] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.520787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.527681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb34bc-7b39-46ab-957a-6a24af84b67b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.530540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664dd27-704b-47ba-9e73-e43cee34742a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.532483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.533312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.533546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.533854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] PATH_INFO: `/v3/projects/0a2594d3f33e4ca59a8b955ecd11daed/users/fe7c213c46c34780a387aca26b779a0d/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.535326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.535977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.536260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.536538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] PATH_INFO: `/v3/projects/59ebfa9c85224499947c24bde556e9cc/users/17daef08b7964e62973d7ebf5ed2f508/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.549329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2594d3f33e4ca59a8b955ecd11daed, user_id=fe7c213c46c34780a387aca26b779a0d, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.552229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031bd2f-bc27-4806-91b3-babd98dbd3dd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.552394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ebfa9c85224499947c24bde556e9cc, user_id=17daef08b7964e62973d7ebf5ed2f508, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.555128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.565660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1075/2140] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:09 2022] PUT /identity/v3/projects/0a2594d3f33e4ca59a8b955ecd11daed/users/fe7c213c46c34780a387aca26b779a0d/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:09.567430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17770-ee2e-4866-85ad-3031b982262d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.567713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17770-ee2e-4866-85ad-3031b982262d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.567989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17770-ee2e-4866-85ad-3031b982262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.569642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1066/2141] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:09 2022] PUT /identity/v3/projects/59ebfa9c85224499947c24bde556e9cc/users/17daef08b7964e62973d7ebf5ed2f508/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:09.570547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.601887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-35b17770-ee2e-4866-85ad-3031b982262d None None] MFA Rules not processed for user `8b7c32f5b53f41368ba22c95ea7b2011`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:09.603118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-35b17770-ee2e-4866-85ad-3031b982262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:09.639888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519edef-0286-4327-b5c8-5301fedda9c3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.644562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.645210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.645485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.645824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.646514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.648439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.658456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1067/2142] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.659270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.669276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1076/2143] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:09.670763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.671056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.671328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.672892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb196d5e-cec3-4f29-a632-f5d134225097 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.681043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.681946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.682355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.682756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.683558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.685751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa0d6-3ec3-4981-b8a2-ba76bea7facd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.700843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1068/2144] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:09.702564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.703025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.703186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] MFA Rules not processed for user `fe7c213c46c34780a387aca26b779a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:09.703665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.704417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:09.735726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] MFA Rules not processed for user `17daef08b7964e62973d7ebf5ed2f508`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:09.736952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:09.772562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1077/2145] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:09.773402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.778618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:09.786324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba9cd-4dc1-4100-a489-14f3c6c7d816 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.788747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:09.791910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.792584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.792861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.793141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] PATH_INFO: `/v3/projects/b3ae471cdde94ae9bede2747d66de4c6/users/a1d8a69cdfaf46ebb0838b429d5b72d4/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.812228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1069/2146] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:09.813851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae471cdde94ae9bede2747d66de4c6, user_id=a1d8a69cdfaf46ebb0838b429d5b72d4, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.814184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.815648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:09.816542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.837724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1078/2147] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:09 2022] PUT /identity/v3/projects/b3ae471cdde94ae9bede2747d66de4c6/users/a1d8a69cdfaf46ebb0838b429d5b72d4/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:09.839104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:09.889165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52186192-81ff-46cd-8cb8-0de55327e14e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.905458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e39a41-a18c-4209-b285-0976960a910e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:09.911271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.911936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.912257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.912710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] PATH_INFO: `/v3/projects/aa6929ebd282417cb6ba149f50281240/users/7790a2ef47644d62863aafc890d03249/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.928510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6929ebd282417cb6ba149f50281240, user_id=7790a2ef47644d62863aafc890d03249, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:09.931217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:09.939463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:09.940099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:09.940378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:09.940657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:09.945116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1079/2148] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:09 2022] PUT /identity/v3/projects/aa6929ebd282417cb6ba149f50281240/users/7790a2ef47644d62863aafc890d03249/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:09.945962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.001293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.002807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2df5-2dc3-4d97-bc49-9cb636cc1375 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.012250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1070/2149] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:10.015162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581c6d-d39d-4afb-a8d5-45fd69241091 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.015162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581c6d-d39d-4afb-a8d5-45fd69241091 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.015162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581c6d-d39d-4afb-a8d5-45fd69241091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.015446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbc022-1557-4f53-be0f-fcf653b40972 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.026924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.027698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.027981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.028248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.029945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.031960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.049393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-69581c6d-d39d-4afb-a8d5-45fd69241091 None None] MFA Rules not processed for user `a1d8a69cdfaf46ebb0838b429d5b72d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:10.050666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-69581c6d-d39d-4afb-a8d5-45fd69241091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.064937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1080/2150] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:09 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:10.065676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.069117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.103932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:10.132641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1071/2151] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:10.136654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:10.136757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.136757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.136757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.140668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e7c5-bd68-456c-86e4-3bf6d3cf20f3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.146628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.148237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.148521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.148829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.160055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:10.161966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.247320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1072/2152] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:10.248565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.261733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5067d1-53e6-46e5-a677-e923bbe9fea7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.267773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.268523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.268890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.269248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.270056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.272119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.294918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.296957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.298700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1073/2153] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:10 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:10.299515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.302056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.309029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1081/2154] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:10.310409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.331174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94b6f-840e-40d8-a850-6fcd5130496c service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.338431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.339630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.340148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.340561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.341686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.343142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.355546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1082/2155] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:10.356528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.367341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c584c1-8f2e-4fae-bfb1-8d1ada558e35 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.371393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.371997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.372278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.372562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.385620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c326f4-015a-4acc-8e8e-62c4b0af2207 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.389330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.389986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.390260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.390557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.401859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.403428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.415111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1074/2156] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:10.416201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.429067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ee51-9d89-4985-b05a-ac36f1327d22 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.435686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.436424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.436704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.436977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] PATH_INFO: `/v3/users/17daef08b7964e62973d7ebf5ed2f508` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.438610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17daef08b7964e62973d7ebf5ed2f508)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.441002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.442058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.443491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.453686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1083/2157] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:10.466391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.482762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615eb76-a61a-47c4-b0f9-8cd8d053c48f service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.489396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.490118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.490412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.491228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.491854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.493538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.515973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1084/2158] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:10.515973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.523425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17daef08b7964e62973d7ebf5ed2f508'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.526746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17daef08b7964e62973d7ebf5ed2f508'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.527762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17daef08b7964e62973d7ebf5ed2f508'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.533157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d8af-4013-4305-b67e-ef07646500c2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.538988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.539635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.539918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.540197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] PATH_INFO: `/v3/users/a1d8a69cdfaf46ebb0838b429d5b72d4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.541694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d8a69cdfaf46ebb0838b429d5b72d4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.543842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:10.555015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17daef08b7964e62973d7ebf5ed2f508'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.556940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:10.582975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17daef08b7964e62973d7ebf5ed2f508'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.610013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1075/2159] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:10 2022] DELETE /identity/v3/users/17daef08b7964e62973d7ebf5ed2f508 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:10.614557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.618108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.640400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d8a69cdfaf46ebb0838b429d5b72d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.664401 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d8a69cdfaf46ebb0838b429d5b72d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.694008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:10.697840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d8a69cdfaf46ebb0838b429d5b72d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.712199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d8a69cdfaf46ebb0838b429d5b72d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.712828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7a22a621-33ae-4a7c-a54b-9bc8e0d0ab18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d8a69cdfaf46ebb0838b429d5b72d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:10.732291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1085/2160] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:10 2022] DELETE /identity/v3/users/a1d8a69cdfaf46ebb0838b429d5b72d4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:10.733625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378c13-5a26-450d-b5db-7596b7c1e5ad None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.733926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378c13-5a26-450d-b5db-7596b7c1e5ad None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.734183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378c13-5a26-450d-b5db-7596b7c1e5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.759242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-29378c13-5a26-450d-b5db-7596b7c1e5ad None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:10.760822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-29378c13-5a26-450d-b5db-7596b7c1e5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.780223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2165-941d-4451-b501-d0c8636c210c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:10.869988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1086/2161] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:10.880715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.881161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.881781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.913263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:10.914369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:10.914821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:10.915862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] PATH_INFO: `/v3/projects/b9e8944dbd3a4a15b31fb07523943e22/users/537ea5d4c1484cc3927cb69b5b05e5c1/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:10.916175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:10.919072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.939950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1087/2162] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:10.941623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:10.957081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:10.976327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e8944dbd3a4a15b31fb07523943e22, user_id=537ea5d4c1484cc3927cb69b5b05e5c1, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:10.983807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.017042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1076/2163] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:10 2022] PUT /identity/v3/projects/b9e8944dbd3a4a15b31fb07523943e22/users/537ea5d4c1484cc3927cb69b5b05e5c1/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:11.017447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.039574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:11.077380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f49392-3cca-407d-aba5-2298a150b7e3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.159334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.160029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.162289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.162726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.234695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320c0e-ecf8-4a30-84fc-e88343906632 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.240604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.241429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.241908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.242383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.262392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:11.303625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:11.311549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.314450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d4bfa-dae0-41ce-8f1c-f53a7a4d8f13 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.335518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1088/2164] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:11.338288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.338834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.339323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.388236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.391773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.404888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] MFA Rules not processed for user `537ea5d4c1484cc3927cb69b5b05e5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:11.412423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:11.414554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1077/2165] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:11.494996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1089/2166] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:11.500137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.501728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:11.519073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab95d92-3106-4baa-8838-2b769d2b8b00 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.524782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.525526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.525941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.526312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.527044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.529891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.569621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1078/2167] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:11 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:11.570937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.589841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4cbd-9dad-4573-98cf-5ed0ea305134 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.703697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.704524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.704809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.705096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.705753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.711350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.759083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1090/2168] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:11 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:11.762966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.778300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c809c-ede8-436e-ac0c-ae1e1dada7ba admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.784799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.785561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.785881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.786312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.786880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.788580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.801355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1079/2169] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:11.805529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.817537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef906e5-1dd0-495d-a419-1e83e8514850 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.823743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.824631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.825178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.826311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] PATH_INFO: `/v3/projects/1d1110b9e90b4164a28e995242e99e9d/users/2aebf4ad11854f6d89583fb53ee94a0f/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.845552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1110b9e90b4164a28e995242e99e9d, user_id=2aebf4ad11854f6d89583fb53ee94a0f, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.848853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.866260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1091/2170] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:11 2022] PUT /identity/v3/projects/1d1110b9e90b4164a28e995242e99e9d/users/2aebf4ad11854f6d89583fb53ee94a0f/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:11.870194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:11.883825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0cf1-4401-4260-8209-d3c06a9f0088 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:11.970470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:11.971103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:11.971822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:11.971822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:11.972300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:11.973990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a39334-b86a-4a54-9b36-c5302d672833 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:11.985644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1080/2171] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:11.989352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:12.000365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086e204-bea3-4932-b04e-f125dc0cbd9f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:12.007356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:12.008118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:12.008398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:12.008730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] PATH_INFO: `/v3/projects/1d1110b9e90b4164a28e995242e99e9d/users/2aebf4ad11854f6d89583fb53ee94a0f/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:12.010882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1110b9e90b4164a28e995242e99e9d, user_id=2aebf4ad11854f6d89583fb53ee94a0f, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:12.013670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:12.029513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1092/2172] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:11 2022] PUT /identity/v3/projects/1d1110b9e90b4164a28e995242e99e9d/users/2aebf4ad11854f6d89583fb53ee94a0f/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:12.048066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:12.048937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:12.049518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:12.096803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] MFA Rules not processed for user `2aebf4ad11854f6d89583fb53ee94a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:12.100161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:12.197348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1081/2173] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:12.209053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:12.211566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:12.267836 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:12.314356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e4d19-b01d-4d88-b2df-aed2ef0b6dc0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:12.319322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:12.319965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:12.320248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:12.320522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:12.322427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:12.425357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:12.427470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:12.445035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1093/2174] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:12.455145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:12.457310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:12.492286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:12.565771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95efb4-d281-45e7-a351-e0bfda18f172 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:12.574386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:12.575277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:12.575957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:12.576305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] PATH_INFO: `/v3/projects/59ebfa9c85224499947c24bde556e9cc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:12.578297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ebfa9c85224499947c24bde556e9cc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:12.591049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:12.663315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ebfa9c85224499947c24bde556e9cc', 'request_id': 'req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:12.673646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ebfa9c85224499947c24bde556e9cc', 'request_id': 'req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:12.695184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ebfa9c85224499947c24bde556e9cc', 'request_id': 'req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:12.725251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] The token cache is being invalidate because project 59ebfa9c85224499947c24bde556e9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:12.725725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:12.727572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cbcb7c4b-b763-42c6-8174-89d41a0e6d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:12.738266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:12.740293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:12.744009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1082/2175] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:12 2022] DELETE /identity/v3/projects/59ebfa9c85224499947c24bde556e9cc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:12.767769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f267-1a8f-438c-8b4d-8896d72a04ce service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:12.985630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:12.986296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:12.986584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:12.986919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] PATH_INFO: `/v3/projects/b3ae471cdde94ae9bede2747d66de4c6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:12.990545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ae471cdde94ae9bede2747d66de4c6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:12.993059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.077507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ae471cdde94ae9bede2747d66de4c6', 'request_id': 'req-412f1e73-f397-44c6-a649-110480e62895'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:13.098685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ae471cdde94ae9bede2747d66de4c6', 'request_id': 'req-412f1e73-f397-44c6-a649-110480e62895'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:13.116673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ae471cdde94ae9bede2747d66de4c6', 'request_id': 'req-412f1e73-f397-44c6-a649-110480e62895'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:13.166782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] The token cache is being invalidate because project b3ae471cdde94ae9bede2747d66de4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:13.167090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:13.168448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-412f1e73-f397-44c6-a649-110480e62895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:13.173451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1094/2176] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:12 2022] DELETE /identity/v3/projects/b3ae471cdde94ae9bede2747d66de4c6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:13.175245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.175671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.176052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.203600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:13.205097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:13.338089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1083/2177] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:13.338930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.339391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.339695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.375229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:13.377681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:13.387531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1095/2178] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:13.391898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.405199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb9aef-5717-49af-b302-8018d0a867c5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.412357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.412933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.413336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.413724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.414570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:13.414884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.417305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.431957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1084/2179] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:13.436520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.450763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5481b-7b94-4e0a-80a4-ab3f78a7491c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.456243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.458334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.458731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.459051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.459873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.462533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.469864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:13.497139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1096/2180] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:13 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:13.501303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.512724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a4d9-daba-4e19-ba4f-12aeaed90837 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.545965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:13.597373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.598056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.598338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.598616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.600010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.601992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.633233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1085/2181] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:13 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:13.637260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.648823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296525-37d9-4a8c-9bc7-e15e0fb8168d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.656110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.657058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.657414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.657798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.658522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.660196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.676589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1097/2182] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:13.677913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.698763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6669-5f40-44f0-8a9a-bc651ab6e102 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.706173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.707706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.708093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.708464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.709230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.711147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a6340-7f31-4d84-aed6-b9d28d2322b7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.751717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1086/2183] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:13.753078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.763856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262caa24-f0bf-4bf1-a8a0-b5590c763a9a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:13.775013 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:13.775759 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.775983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.776267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] PATH_INFO: `/v3/projects/740832491e454b40a6cd87be4ca40384/users/b51990c984e24a18a85534b121f3d9a0/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.797081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740832491e454b40a6cd87be4ca40384, user_id=b51990c984e24a18a85534b121f3d9a0, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:13.801688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:13.817883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1098/2184] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:13 2022] PUT /identity/v3/projects/740832491e454b40a6cd87be4ca40384/users/b51990c984e24a18a85534b121f3d9a0/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:13.823001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:13.823376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:13.823702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:13.846457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:13.847684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:13.940534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1087/2185] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:13.954043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:13.955927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:14.028112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c245bb7-1590-4251-9972-3314db31ff87 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:14.032209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:14.033161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:14.033687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:14.034172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:14.045361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:14.047232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:14.060688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1099/2186] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:14.420491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:14.422213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:14.515439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d141cc-1f74-43ac-89dc-589c9c8d56f7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:14.521534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:14.522648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:14.523229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:14.523785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:14.526045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:14.655326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:14.656981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9456da3-d195-4dec-ac40-6cfc769edd57 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:14.670320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1088/2187] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:15.751141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:15.761479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e7d7-11ef-43b8-80af-bae4db13038a service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:15.765145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:15.765777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:15.766061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:15.766335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:15.768075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:15.863110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:15.865235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:15.878074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1100/2188] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:17.581550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c970-a94b-4fe5-9ed8-45d5cbac4b21 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:17.582175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c970-a94b-4fe5-9ed8-45d5cbac4b21 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:17.582577 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c970-a94b-4fe5-9ed8-45d5cbac4b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:17.594488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:17.637544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-30d9c970-a94b-4fe5-9ed8-45d5cbac4b21 None None] MFA Rules not processed for user `7790a2ef47644d62863aafc890d03249`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:17.639168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-30d9c970-a94b-4fe5-9ed8-45d5cbac4b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:17.741640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1089/2189] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:17.760302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:17.762180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:17.823599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:17.862296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fc757-f369-43bc-8a0f-3c3de7bcbdec service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:17.881173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:17.881995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:17.882408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:17.882683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:17.892084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:17.893787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:17.905015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1101/2190] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:20.484629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5decc5-384f-4ec0-b904-f278fdd1eb41 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:20.485371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5decc5-384f-4ec0-b904-f278fdd1eb41 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:20.485828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5decc5-384f-4ec0-b904-f278fdd1eb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:20.512116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4e5decc5-384f-4ec0-b904-f278fdd1eb41 None None] MFA Rules not processed for user `7790a2ef47644d62863aafc890d03249`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:20.514550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4e5decc5-384f-4ec0-b904-f278fdd1eb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:20.537542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1090/2191] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:20.745391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:20.747623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:20.765050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237c9c0-59f7-4f01-ac55-2d4dfc876de1 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:20.778254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:20.779146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:20.782691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:20.783142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] PATH_INFO: `/v3/users/fe7c213c46c34780a387aca26b779a0d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:20.793986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7c213c46c34780a387aca26b779a0d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:20.796165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:20.917417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7c213c46c34780a387aca26b779a0d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:20.929918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7c213c46c34780a387aca26b779a0d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:20.959812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7c213c46c34780a387aca26b779a0d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:20.976094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7c213c46c34780a387aca26b779a0d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:20.976788 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7c213c46c34780a387aca26b779a0d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:20.983863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1102/2192] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:20 2022] DELETE /identity/v3/users/fe7c213c46c34780a387aca26b779a0d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:21.002400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc8ce-573f-447d-9893-76bb9ba4aa5f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:21.002938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc8ce-573f-447d-9893-76bb9ba4aa5f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:21.004715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bc8ce-573f-447d-9893-76bb9ba4aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:21.050473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-484bc8ce-573f-447d-9893-76bb9ba4aa5f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:21.052436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-484bc8ce-573f-447d-9893-76bb9ba4aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:21.171910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1091/2193] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:21.185088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:21.186656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:21.342222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767cb90-ca08-4fc2-abfb-001a506e5464 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:21.350877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:21.351541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:21.351825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:21.352145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:21.378333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:21.380412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:21.394393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1103/2194] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:21.980889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:21.981772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:21.982194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.022327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] MFA Rules not processed for user `b51990c984e24a18a85534b121f3d9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:22.023862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:22.169332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1092/2195] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:22.174119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.175803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:22.212070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e57fd0-cf9f-4111-9611-29890a046e53 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.222074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:22.224922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:22.225283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:22.225775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.227553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:22.241503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:22.313713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1104/2196] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:22 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:22.319345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.355988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140b5a0-460c-48df-b59c-2684c4a2d1d5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.390123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:22.524093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:22.524733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:22.525016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:22.525290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.525975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:22.529148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:22.561360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.561982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1093/2197] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:22 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:22.566114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.568126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:22.586351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5b66-561c-4f82-8110-abb7469f2633 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.596594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:22.597361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:22.597791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:22.598213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.598947 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:22.612275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:22.634826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7257-30fa-4a6d-b90e-b76f8e77de39 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.637134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1094/2198] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:22.638140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.664952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337c30-10d1-4cc8-beb6-d91ff0d46d29 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.667841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:22.668954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:22.669467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:22.670038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] PATH_INFO: `/v3/projects/0a2594d3f33e4ca59a8b955ecd11daed` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.672679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2594d3f33e4ca59a8b955ecd11daed)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:22.676997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:22.679049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:22.680165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:22.680609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:22.681354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] PATH_INFO: `/v3/projects/03bad1c87f1c487a9fad40a27cfec1ae/users/ad0d139854c6485889d8a3adbb61377f/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:22.724861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03bad1c87f1c487a9fad40a27cfec1ae, user_id=ad0d139854c6485889d8a3adbb61377f, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:22.728357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:22.759749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1095/2199] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:22 2022] PUT /identity/v3/projects/03bad1c87f1c487a9fad40a27cfec1ae/users/ad0d139854c6485889d8a3adbb61377f/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:22.764475 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:22.774409 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.859593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2594d3f33e4ca59a8b955ecd11daed', 'request_id': 'req-f618e893-a690-4861-b9ce-5448be498dc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:22.880878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2594d3f33e4ca59a8b955ecd11daed', 'request_id': 'req-f618e893-a690-4861-b9ce-5448be498dc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:22.888692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc7768-ef82-4299-81fb-17531dcfc88f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:22.895711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2594d3f33e4ca59a8b955ecd11daed', 'request_id': 'req-f618e893-a690-4861-b9ce-5448be498dc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:22.927828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] The token cache is being invalidate because project 0a2594d3f33e4ca59a8b955ecd11daed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:22.928058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:22.929260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:22.931261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1105/2200] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:22 2022] DELETE /identity/v3/projects/0a2594d3f33e4ca59a8b955ecd11daed => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 21:56:22.939262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:22.941055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:23.043204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:23.044420 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:23.045083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:23.045973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:23.047101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:23.052101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:23.077669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1096/2201] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:23.081216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:23.081472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618e893-a690-4861-b9ce-5448be498dc5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:23.086893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:23.096711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:23.099221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:23.099717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:23.100148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] PATH_INFO: `/v3/users/537ea5d4c1484cc3927cb69b5b05e5c1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:23.101686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537ea5d4c1484cc3927cb69b5b05e5c1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:23.104008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:23.129242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604391fc-c319-488d-8467-0e0011ef860c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:23.162634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:23.163403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:23.163974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:23.166140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] PATH_INFO: `/v3/projects/03bad1c87f1c487a9fad40a27cfec1ae/users/ad0d139854c6485889d8a3adbb61377f/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:23.187097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03bad1c87f1c487a9fad40a27cfec1ae, user_id=ad0d139854c6485889d8a3adbb61377f, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:23.203878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:23.204386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537ea5d4c1484cc3927cb69b5b05e5c1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:23.220462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537ea5d4c1484cc3927cb69b5b05e5c1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:23.226448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:23.239642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537ea5d4c1484cc3927cb69b5b05e5c1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:23.241652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1097/2202] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:23 2022] PUT /identity/v3/projects/03bad1c87f1c487a9fad40a27cfec1ae/users/ad0d139854c6485889d8a3adbb61377f/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:23.260412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537ea5d4c1484cc3927cb69b5b05e5c1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:23.261191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537ea5d4c1484cc3927cb69b5b05e5c1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:23.272904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1106/2203] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:22 2022] DELETE /identity/v3/users/537ea5d4c1484cc3927cb69b5b05e5c1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:23.829657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:23.833270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:23.915575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c798c-eba2-40a3-93ca-b671c0c4eeb6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:24.007698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:24.008772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:24.009219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:24.009684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] PATH_INFO: `/v3/projects/b9e8944dbd3a4a15b31fb07523943e22` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:24.011510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e8944dbd3a4a15b31fb07523943e22)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:24.013964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:24.088644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e8944dbd3a4a15b31fb07523943e22', 'request_id': 'req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.095755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e8944dbd3a4a15b31fb07523943e22', 'request_id': 'req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.105076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e8944dbd3a4a15b31fb07523943e22', 'request_id': 'req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.118762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] The token cache is being invalidate because project b9e8944dbd3a4a15b31fb07523943e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:24.119186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.120934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.126185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1098/2204] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:23 2022] DELETE /identity/v3/projects/b9e8944dbd3a4a15b31fb07523943e22 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:24.131088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:24.141862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fb9ee-6f01-450b-b25b-8b36acb27b45 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:24.173528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:24.216643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:24.217208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:24.217494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:24.217888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] PATH_INFO: `/v3/users/2aebf4ad11854f6d89583fb53ee94a0f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:24.219252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aebf4ad11854f6d89583fb53ee94a0f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:24.221233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:24.299613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aebf4ad11854f6d89583fb53ee94a0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.313252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aebf4ad11854f6d89583fb53ee94a0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.352738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aebf4ad11854f6d89583fb53ee94a0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.366552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aebf4ad11854f6d89583fb53ee94a0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.367354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fdfb9621-43b3-4a6a-8243-53312eb4cc88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aebf4ad11854f6d89583fb53ee94a0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:24.376385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1107/2205] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:24 2022] DELETE /identity/v3/users/2aebf4ad11854f6d89583fb53ee94a0f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:24.939202 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:24.940879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:24.964497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273b8a2-8d58-4318-a0e7-79355b7dc0f0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.016395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:25.058131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.059000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.059310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.059541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] PATH_INFO: `/v3/projects/1d1110b9e90b4164a28e995242e99e9d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.060723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1110b9e90b4164a28e995242e99e9d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.062949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:25.163229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1110b9e90b4164a28e995242e99e9d', 'request_id': 'req-862bf354-bf6e-480b-bea5-585ef453aa58'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:25.171471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1110b9e90b4164a28e995242e99e9d', 'request_id': 'req-862bf354-bf6e-480b-bea5-585ef453aa58'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:25.197732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1110b9e90b4164a28e995242e99e9d', 'request_id': 'req-862bf354-bf6e-480b-bea5-585ef453aa58'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:25.213975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] The token cache is being invalidate because project 1d1110b9e90b4164a28e995242e99e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:25.214387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:25.215159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-862bf354-bf6e-480b-bea5-585ef453aa58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:25.217136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1099/2206] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:24 2022] DELETE /identity/v3/projects/1d1110b9e90b4164a28e995242e99e9d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:25.221877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.225455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.225942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.269286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:25.275868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:25.419141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1108/2207] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:25.419339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.419622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.419904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.456171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:25.457978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:25.487083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1100/2208] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:25.489069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:25.501208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c0097-f85a-4eac-a4dd-70a5f2f115dc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.508415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.509103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.509588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.509976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.510964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:25.511090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.513096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:25.542090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:25.545945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1109/2209] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:25.564318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4a2e6-bf73-4e20-afc7-af2af14a9475 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.585489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.586293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.586851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.588769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.590010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.595591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:25.669839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1101/2210] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:25 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:25.675039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:25.701146 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93588-04b2-47fa-b2e6-07b71a44dad8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.748898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:25.823875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.824641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.824957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.825230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.825957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.828635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:25.894176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1110/2211] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:25 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 21:56:25.899933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:25.916404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804ead-899e-410d-8bd0-953943e630e5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.924284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.925164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.925581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.926851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.928120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.930114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:25.946280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1102/2212] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:25.950160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:25.971895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779dd87-24d1-4fb9-a74e-97b40ce0abdd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:25.978895 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:25.979349 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:25.979688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:25.980102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:25.980850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:25.982780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f12ba-aa4c-4a95-808f-ab251b1859be admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:26.000215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1111/2213] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:26.001332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:26.015914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc479a0-bf06-4a70-abf0-93af3d97671e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:26.030985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:26.035842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:26.036252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:26.036535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] PATH_INFO: `/v3/projects/3b157ed00f1548f1af769106e03cf9b5/users/6a40fd568da34b26bd93b8ff0ea92812/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:26.078422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b157ed00f1548f1af769106e03cf9b5, user_id=6a40fd568da34b26bd93b8ff0ea92812, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:26.082523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:26.104851 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1103/2214] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:26 2022] PUT /identity/v3/projects/3b157ed00f1548f1af769106e03cf9b5/users/6a40fd568da34b26bd93b8ff0ea92812/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:26.109775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8131f5-b4b9-4eb8-acd0-df59647e5580 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:26.110062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8131f5-b4b9-4eb8-acd0-df59647e5580 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:26.110339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8131f5-b4b9-4eb8-acd0-df59647e5580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:26.154057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5c8131f5-b4b9-4eb8-acd0-df59647e5580 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:26.155649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5c8131f5-b4b9-4eb8-acd0-df59647e5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:26.259644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1112/2215] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:26.269733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:26.273261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:26.350345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68209397-2bd2-415a-b895-00ffad6fb98c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:26.356916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:26.357547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:26.357913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:26.358186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:26.371788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:26.373251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:26.393530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1104/2216] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:27.586964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900fe6a-8ce8-4a8b-9f0f-75b2a511e601 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:27.588022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900fe6a-8ce8-4a8b-9f0f-75b2a511e601 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:27.590111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900fe6a-8ce8-4a8b-9f0f-75b2a511e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:27.693536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c900fe6a-8ce8-4a8b-9f0f-75b2a511e601 None None] MFA Rules not processed for user `6a40fd568da34b26bd93b8ff0ea92812`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:27.704301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c900fe6a-8ce8-4a8b-9f0f-75b2a511e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:27.842211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1113/2217] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:27.853689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:27.855808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:27.869056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:27.974907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccfa-3e81-4ad8-9efb-70aba41f3303 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:27.993090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:28.002145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:28.009886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:28.010469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:28.025499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:28.027963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bebf1e-38d1-4a2e-bfbd-c55d1a7be316 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:28.046725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1105/2218] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:29.981703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:29.982749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:29.983201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:30.008256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] MFA Rules not processed for user `6a40fd568da34b26bd93b8ff0ea92812`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:30.009520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:30.022829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1114/2219] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:30.929720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f34a-1523-4c09-b93c-bc2bd7db0287 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:30.930528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f34a-1523-4c09-b93c-bc2bd7db0287 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:30.931021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f34a-1523-4c09-b93c-bc2bd7db0287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:30.970624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-21f0f34a-1523-4c09-b93c-bc2bd7db0287 None None] MFA Rules not processed for user `ad0d139854c6485889d8a3adbb61377f`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:30.972036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-21f0f34a-1523-4c09-b93c-bc2bd7db0287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:31.088617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1106/2220] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:31.101052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:31.102538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:31.181025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:31.192752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca62ca0-2e97-4e85-a03c-f2e48062913b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:31.202801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:31.203837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:31.204552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:31.204828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:31.206924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:31.290701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:31.292830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:31.314015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1115/2221] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:31.466119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3a61-af23-4c74-82f6-857b199c78f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:31.466557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3a61-af23-4c74-82f6-857b199c78f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:31.466912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3a61-af23-4c74-82f6-857b199c78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:31.490883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a0dd3a61-af23-4c74-82f6-857b199c78f7 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:31.492171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a0dd3a61-af23-4c74-82f6-857b199c78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:31.569981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1107/2222] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:56:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:31.616156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:31.618328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:31.718967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200303f-ce42-475d-b830-fbccdb5725da service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:31.727447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:31.728284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:31.728756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:31.728994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:31.744893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:31.746634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bafa22-8bfa-465a-98bb-12186ad3e3f7 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:31.755488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1116/2223] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:56:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:56:41.553949 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=82 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:41.751837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=63 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:43.567625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:43.568315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:43.568315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:43.594437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] MFA Rules not processed for user `b51990c984e24a18a85534b121f3d9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:43.595760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:43.609398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1108/2224] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:56:45.733581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:45.734120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:45.734226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:45.754319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:45.756123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:45.769990 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1117/2225] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:56:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:45.783951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:45.797045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa18f-6918-4fcc-b8de-8379f09e62b9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:45.804334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:45.805227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:45.805503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:45.805954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:45.815757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:45.818161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea6d0a-68ab-433e-af4a-104484ea077b service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:45.826742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1109/2226] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:56:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:56:46.453191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:46.456122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:46.473080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43d851-9cd5-441a-b1b7-50e0d4b7659c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:46.477807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:46.478446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:46.478726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:46.479000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] PATH_INFO: `/v3/users/8b7c32f5b53f41368ba22c95ea7b2011` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:46.480149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7c32f5b53f41368ba22c95ea7b2011)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:46.482119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:46.534334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7c32f5b53f41368ba22c95ea7b2011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:46.541275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7c32f5b53f41368ba22c95ea7b2011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:46.554363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7c32f5b53f41368ba22c95ea7b2011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:46.562504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7c32f5b53f41368ba22c95ea7b2011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:46.563323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7c32f5b53f41368ba22c95ea7b2011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:46.570225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1118/2227] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:56:46 2022] DELETE /identity/v3/users/8b7c32f5b53f41368ba22c95ea7b2011 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:46.574747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:46.575035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:46.575311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:46.596300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:46.597548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:46.731963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1110/2228] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:46.809946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:46.811996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:46.934714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103271e-ff68-43a3-8eac-24989cebfe18 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:46.953084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:46.953915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:46.954300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:46.954676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:46.968260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:46.969757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda8387-ce4e-4b27-9be1-307ba517fbcd service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:46.982582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1119/2229] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:56:47.661706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:47.663578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:47.686173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ac2ef-aef4-4202-8bc3-402e445f7543 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:47.711645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:47.712315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:47.722154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:47.722836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] PATH_INFO: `/v3/projects/873d8d780a644e38b303cc76e17f3231` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:47.730593 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=873d8d780a644e38b303cc76e17f3231)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:47.735583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:47.874474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873d8d780a644e38b303cc76e17f3231', 'request_id': 'req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:47.888615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873d8d780a644e38b303cc76e17f3231', 'request_id': 'req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:47.914549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873d8d780a644e38b303cc76e17f3231', 'request_id': 'req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:47.952314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] The token cache is being invalidate because project 873d8d780a644e38b303cc76e17f3231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:56:47.952624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:47.953346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b66e3298-ddfd-49d8-8865-7e4c0bea98fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:56:47.957642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1111/2230] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:56:47 2022] DELETE /identity/v3/projects/873d8d780a644e38b303cc76e17f3231 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:47.960654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:47.961196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:47.961196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.014951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:48.020073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:48.128074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1120/2231] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:48.129542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.129864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.130918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.164182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:48.165628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:48.178845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1112/2232] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:48.180218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.196640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40d09-c329-4926-9ce0-a3de75655483 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.202078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.202755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.203304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.203589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.204252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 21:56:48.204557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.207319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.210955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:48.225643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1121/2233] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 21:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 21:56:48.236298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.249276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b521f12-6a1e-45b3-9714-0129e52b5c58 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.274959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.276307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.276591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.276910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.277686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.280351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.343436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1113/2234] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 21:56:48 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:48.347576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.365548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7959c-c0fc-4c52-a5a5-00772fe8641b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.455104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.457516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.459221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.459616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.461314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.473462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.531905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1122/2235] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 21:56:48 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 21:56:48.536053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.573812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be544f4-b943-4472-a22e-0fe817fa8ddd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.601646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.602618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.603205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.603476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.612053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.617948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.653384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1114/2236] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:48.654050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.667297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a7fa-f665-4efd-92dd-91705a8988f0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.673561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.673891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.674434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.674660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.676118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.678506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1f93c-70e2-409d-b392-dcb9847bd925 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.707981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1123/2237] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 21:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 21:56:48.712197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:48.727552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822d39a-f57c-4483-966a-72c2f5707d1d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:48.735138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:48.735827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.736106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.737646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] PATH_INFO: `/v3/projects/c7a2688ac4b541baa168b2930b11af2c/users/8c97093d31e34a5cb778f3922f5f81ef/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.772583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a2688ac4b541baa168b2930b11af2c, user_id=8c97093d31e34a5cb778f3922f5f81ef, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:48.781364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:48.820524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:48.824809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1115/2238] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 21:56:48 2022] PUT /identity/v3/projects/c7a2688ac4b541baa168b2930b11af2c/users/8c97093d31e34a5cb778f3922f5f81ef/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:56:48.838671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:48.839016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:48.839309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:48.882626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] MFA Rules not processed for user `8c97093d31e34a5cb778f3922f5f81ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:56:48.884097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:49.039855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1124/2239] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:56:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:56:49.054765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:56:49.057806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:56:49.150255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32e91-6bbe-41eb-a8cb-71f6c42a544a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:56:49.153983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:56:49.154683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:56:49.155043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:56:49.155377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:56:49.164894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:56:49.166450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daea3c5-2a85-4642-97ca-1cf43de57b7d service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:56:49.175783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1116/2240] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:56:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:56:59.036477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:56:59.171848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=26 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:00.299095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:00.437833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950a77-f0e0-4bfe-b899-76ffecdbffdf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:00.454837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:00.456271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:00.457438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:00.458291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] PATH_INFO: `/v3/users/8c97093d31e34a5cb778f3922f5f81ef` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:00.459730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c97093d31e34a5cb778f3922f5f81ef)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:00.462124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:00.552083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c97093d31e34a5cb778f3922f5f81ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:00.560896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c97093d31e34a5cb778f3922f5f81ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:00.579809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c97093d31e34a5cb778f3922f5f81ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:00.590677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c97093d31e34a5cb778f3922f5f81ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:00.591290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c97093d31e34a5cb778f3922f5f81ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:00.606913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1125/2241] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 21:57:00 2022] DELETE /identity/v3/users/8c97093d31e34a5cb778f3922f5f81ef => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:57:00.609325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:00.609857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:00.610156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:00.633564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:00.634983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:00.700488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1117/2242] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:00.711014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:00.712442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:00.780145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd53-d013-4647-8189-3b8719fce8b7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:00.783936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:00.784735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:00.784978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:00.785350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:00.795853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:00.797437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ce48-46f2-4e6a-bf8d-9920e5c8daad service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:00.810084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1126/2243] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:57:01.179838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:01.181310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:01.194135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1297a-7fe3-40d3-a9f9-f5c5c895ac83 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:01.198732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:01.199362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:01.199640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:01.199917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] PATH_INFO: `/v3/projects/c7a2688ac4b541baa168b2930b11af2c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:01.201018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a2688ac4b541baa168b2930b11af2c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:01.202995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:01.252464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a2688ac4b541baa168b2930b11af2c', 'request_id': 'req-74891408-61c3-45f3-b4ce-a8c54246b964'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:01.257965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a2688ac4b541baa168b2930b11af2c', 'request_id': 'req-74891408-61c3-45f3-b4ce-a8c54246b964'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:01.265363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a2688ac4b541baa168b2930b11af2c', 'request_id': 'req-74891408-61c3-45f3-b4ce-a8c54246b964'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:01.276936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] The token cache is being invalidate because project c7a2688ac4b541baa168b2930b11af2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 21:57:01.277226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:01.277900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 21:57:01.279512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1118/2244] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 21:57:01 2022] DELETE /identity/v3/projects/c7a2688ac4b541baa168b2930b11af2c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 21:57:06.698438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1747f6-33cf-4a3f-a06c-b9cd91f920f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:06.699039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1747f6-33cf-4a3f-a06c-b9cd91f920f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:06.699039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1747f6-33cf-4a3f-a06c-b9cd91f920f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:06.719738 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9c1747f6-33cf-4a3f-a06c-b9cd91f920f6 None None] MFA Rules not processed for user `42072422001c4f1684f68baa25517a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:06.721108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c1747f6-33cf-4a3f-a06c-b9cd91f920f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:06.739818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:06.781413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1127/2245] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:57:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:11.273822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:16.773801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=15 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:20.349986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:20.352743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:20.429134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891408-61c3-45f3-b4ce-a8c54246b964 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:20.433672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:20.434490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:20.434922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:20.435496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:20.438588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:20.522895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:20.524718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:20.539070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1119/2246] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:57:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:57:21.600448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497831a-02c8-4815-b51d-c5bf778fae78 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:21.601103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497831a-02c8-4815-b51d-c5bf778fae78 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:21.601103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497831a-02c8-4815-b51d-c5bf778fae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:21.626062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8497831a-02c8-4815-b51d-c5bf778fae78 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:21.627424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8497831a-02c8-4815-b51d-c5bf778fae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:21.711042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1128/2247] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:57:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:21.725287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:21.726808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:21.844519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8face537-af7e-4d1d-9ba2-a11e88a5d860 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:21.851150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:21.851803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:21.852664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:21.852664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:21.861708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:21.863379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b479f-6b6d-4397-b565-53f3a2c3cdbd service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:21.872201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1120/2248] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:57:31.698700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=25 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:31.870260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=82 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:43.369255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.369852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.369936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.371223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1129/2249] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:57:43.378187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c346-94c7-4ed3-8ec7-a9915d565195 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.378692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c346-94c7-4ed3-8ec7-a9915d565195 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.378797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04c346-94c7-4ed3-8ec7-a9915d565195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.398474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ff04c346-94c7-4ed3-8ec7-a9915d565195 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:43.399696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ff04c346-94c7-4ed3-8ec7-a9915d565195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:43.468519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1121/2250] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:57:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:43.480561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:43.482355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:43.544297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85447-54c5-466b-bb19-20fad3268708 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:43.548031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:43.548639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.548909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.549174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.559443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:43.560863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c96c-a5e1-423e-ac05-14992532b69a service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:43.571316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1130/2251] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:57:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:57:43.640876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.641161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.641439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.642824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1122/2252] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:57:43.648054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742df05b-3a9b-40a0-993a-74c8bc5c1270 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.648339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742df05b-3a9b-40a0-993a-74c8bc5c1270 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.648607 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742df05b-3a9b-40a0-993a-74c8bc5c1270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.668540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-742df05b-3a9b-40a0-993a-74c8bc5c1270 None None] MFA Rules not processed for user `2837aa09ce6d4000985f475688f612e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:43.669784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-742df05b-3a9b-40a0-993a-74c8bc5c1270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:43.676986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1131/2253] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:57:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:43.688554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:43.690386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:43.705701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25575f6-a1dc-4801-baa1-f446ee8a75c8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:43.710876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:43.711496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.711780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.713650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.720714 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:43.722608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df81ff-4720-4b90-b288-3649adfa409e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:43.731026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1123/2254] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:57:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:57:43.829104 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9c00-bfa3-43ac-9907-28ee4d3849e0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.829394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9c00-bfa3-43ac-9907-28ee4d3849e0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.829695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9c00-bfa3-43ac-9907-28ee4d3849e0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.831569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1132/2255] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 21:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 21:57:43.837500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51865-c8f5-47cc-953c-9f4a58ff8c58 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.837857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51865-c8f5-47cc-953c-9f4a58ff8c58 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.838127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51865-c8f5-47cc-953c-9f4a58ff8c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.858022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-98a51865-c8f5-47cc-953c-9f4a58ff8c58 None None] MFA Rules not processed for user `d441753da1af4f3c8f3b5fb4088d201b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:43.859133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-98a51865-c8f5-47cc-953c-9f4a58ff8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:43.868628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1124/2256] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 21:57:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:43.879667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9c00-bfa3-43ac-9907-28ee4d3849e0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:43.891098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9c00-bfa3-43ac-9907-28ee4d3849e0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:43.895166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:43.895819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:43.896101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:43.896378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:43.908706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:43.910852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d08bc0-eb99-4d8e-83dd-162e1a2e7985 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:43.922872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1133/2257] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:57:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 21:57:52.814802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:52.815304 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:52.815663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:52.836871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] MFA Rules not processed for user `7790a2ef47644d62863aafc890d03249`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:52.838184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:52.905405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1125/2258] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:57:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:57:53.917991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=35 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:57:55.417816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:55.418996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:55.419448 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:55.440217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:57:55.442714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:57:55.450719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1134/2259] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 21:57:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:57:55.465884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:57:55.478207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98873f6e-584a-4121-aa55-283a26d6a33e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:57:55.483353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:57:55.484120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:57:55.484473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:57:55.484811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:57:55.494388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:57:55.496169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7b10-b937-4772-a479-e77aa416a51d service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:57:55.504411 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1126/2260] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:57:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:58:05.434601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:58:05.501106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=71 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:58:07.270652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:58:07.272151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:58:07.355237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66f43f-8817-4045-ae9b-eefab7a78989 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:58:07.360309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:58:07.360921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:58:07.361194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:58:07.361465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:58:07.365374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:58:07.379834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:58:07.381429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720ee45-860f-44cd-8ad8-ef17456fb0ef service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:58:07.398117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1135/2261] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:58:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:58:16.344626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:58:16.345147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:58:16.345253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:58:16.363942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] MFA Rules not processed for user `6a40fd568da34b26bd93b8ff0ea92812`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:58:16.365148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:58:16.419916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1127/2262] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:58:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:58:17.390566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:58:20.665127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079c8d0-05b8-46cc-8c9f-d23ad1d0cdd5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:58:20.666269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079c8d0-05b8-46cc-8c9f-d23ad1d0cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:58:20.666479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079c8d0-05b8-46cc-8c9f-d23ad1d0cdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:58:20.689814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2079c8d0-05b8-46cc-8c9f-d23ad1d0cdd5 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:58:20.691315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2079c8d0-05b8-46cc-8c9f-d23ad1d0cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:58:20.700605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1136/2263] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 21:58:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 21:58:20.714251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:58:20.730668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdf20b-9153-4ef8-a9e5-d87f7eba131b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:58:20.735564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:58:20.736222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:58:20.736479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:58:20.736910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:58:20.751808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:58:20.755402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:58:20.765055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1128/2264] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 21:58:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 21:58:30.683350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:58:30.761003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=31 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:58:54.742825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:58:54.743526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:58:54.743526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:58:54.763159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] MFA Rules not processed for user `42072422001c4f1684f68baa25517a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 21:58:54.764364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:58:54.771445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1137/2265] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 21:58:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 21:59:04.757987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 21:59:55.052220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:59:55.053645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:59:55.109412 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f08b-ea30-41af-9958-c04be449d9b5 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:59:55.113006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:59:55.113621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:59:55.113913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:59:55.114185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:59:55.115873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:59:55.175371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:59:55.176809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:59:55.186506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1129/2266] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:59:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:59:57.295880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:59:57.297431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:59:57.315308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7c9eb-4cde-4e61-a213-931ff2110a5f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:59:57.320815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:59:57.321430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:59:57.321754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:59:57.322036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:59:57.323832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:59:57.389691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:59:57.391305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:59:57.400811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1138/2267] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:59:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 21:59:59.734064 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 21:59:59.746856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70abd4b-6ec3-4ba5-8693-031ed456e927 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 21:59:59.752214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 21:59:59.752872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 21:59:59.753148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 21:59:59.753418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 21:59:59.755327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 21:59:59.766688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 21:59:59.768292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 21:59:59.778607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1130/2268] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 21:59:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:00:07.396527 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=28 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:00:09.773399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=54 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:00:58.705473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:00:58.719810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2246748-7907-4efd-851f-1c6bfbea42f0 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:00:58.723892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:00:58.724510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:00:58.724787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:00:58.725059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:00:58.727667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:00:58.737882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:00:58.739314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:00:58.748331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1139/2269] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:00:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:01:08.744671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:01:17.990211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:01:17.991632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:01:18.005889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e1662-a5e3-474c-8aea-4f8ce5fb68b5 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:01:18.011451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:01:18.012047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:01:18.012352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:01:18.012628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:01:18.014367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:01:18.025095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:01:18.026717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:01:18.036173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1131/2270] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:01:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:01:28.031474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:01:28.372809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:01:28.382273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea8733-ce36-43a7-911e-7123b493cb25 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:01:28.385824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:01:28.386424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:01:28.386704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:01:28.386989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:01:28.388681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:01:28.400566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:01:28.402020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:01:28.411324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1140/2271] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:01:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:01:31.391823 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:01:31.404297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6dc01-ff63-4756-92e2-8535cea8a262 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:01:31.410322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:01:31.410911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:01:31.411185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:01:31.411449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:01:31.413146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:01:31.471998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:01:31.474315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:01:31.483797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1132/2272] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:01:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:01:38.406878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:01:41.479599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=28 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:03:57.967658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:03:57.969079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:03:58.056057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83a03a-faf3-4385-8e53-f8b0b22857c0 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:03:58.061276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:03:58.061924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:03:58.062206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:03:58.062480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:03:58.071052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:03:58.072615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:03:58.082269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1141/2273] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:03:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:04:08.078196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=37 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:04:36.480575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:04:36.509671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2bd91-6a5e-40e3-bf5d-1427e42b8393 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:04:36.513267 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:04:36.513907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:04:36.514191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:04:36.514466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:04:36.516243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:04:36.548726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:04:36.550450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:04:36.571086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1133/2274] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:04:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:04:46.567908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=27 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:04:55.622248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:04:55.635254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15062f02-28ca-4bb2-84b6-a0f90a4727c1 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:04:55.640684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:04:55.641287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:04:55.641568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:04:55.641927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:04:55.649812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:04:55.651389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:04:55.678311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1142/2275] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:04:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:04:57.674592 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:04:57.686129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12399d79-ae89-4265-92f1-b3e4ea3bc057 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:04:57.692154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:04:57.692768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:04:57.693049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:04:57.693323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:04:57.702670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:04:57.704240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:04:57.731944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1134/2276] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:04:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:05:00.138177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:05:00.149192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874fb7b-b139-423c-a37b-ad8cdb3a151f service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:05:00.155231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:05:00.156174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:05:00.156538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:05:00.156885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:05:00.165465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:05:00.167071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:05:00.195884 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1143/2277] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:05:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:05:07.729235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=13 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:05:10.192744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=26 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:05:18.050972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:05:18.060627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0210-0c77-4def-845b-5d28e0a70d35 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:05:18.064192 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:05:18.064778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:05:18.065053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:05:18.065318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:05:18.067175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:05:18.080718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:05:18.082204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:05:18.093017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1135/2278] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:05:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:05:28.088310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:06:06.690321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:06:06.710395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4679dda-80fa-4ec6-91e3-e310ca8760cf service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:06:06.714063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:06:06.714680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:06:06.714965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:06:06.715236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:06:06.722768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:06:06.724847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:06:06.732856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1144/2279] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:06:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:06:16.729299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:06:17.652392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:06:17.653936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:06:17.680513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944efa0-4e74-463b-8bf6-4b63b1377f20 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:06:17.684236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:06:17.684842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:06:17.685126 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:06:17.685400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:06:17.694054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:06:17.695483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:06:17.704433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1136/2280] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:06:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:06:27.700402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=11 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:06:28.781143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:06:28.790633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f04af3-5e93-4014-bcd2-ae46c91dfd25 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:06:28.794201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:06:28.794797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:06:28.795076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:06:28.795346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:06:28.803784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:06:28.805194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:06:28.833023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1145/2281] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:06:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:06:32.188713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:06:32.198303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25eeed1-f93c-42f8-8ba2-5caba6731042 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:06:32.201888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:06:32.202481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:06:32.202755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:06:32.203022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:06:32.211636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:06:32.213055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:06:32.241151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1137/2282] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:06:38.829880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=13 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:06:42.238054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=13 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:08:59.973049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:08:59.974652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:09:00.043391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5d44a-3e83-4f91-9fdd-94f58ab1bc0d service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:09:00.048328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:09:00.048962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:09:00.049241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:09:00.049803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:09:00.051623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:09:00.063734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:09:00.065309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:09:00.076061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1146/2283] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:08:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:09:10.071821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:09:56.032763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:09:56.102683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62b1a-1a2e-43fa-9ab9-e32dc905172c service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:09:56.108234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:09:56.108904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:09:56.109191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:09:56.109467 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:09:56.111358 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:09:56.183179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:09:56.184755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:09:56.194247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1138/2284] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:09:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:09:57.917370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:09:57.973335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b8d6-92b1-4670-b062-4ac819155991 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:09:57.977491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:09:57.978126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:09:57.978406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:09:57.978677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:09:57.980526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:09:58.049800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:09:58.051236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:09:58.060766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1147/2285] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:09:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:10:00.333910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:10:00.343346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b472c4c-1fad-4d2e-9fff-79ba83c7d721 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:10:00.346991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:10:00.347585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:10:00.347865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:10:00.348138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:10:00.350021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:10:00.420582 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:10:00.422050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:10:00.432464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1139/2286] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:10:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:10:08.056593 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=53 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:10:10.427069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:10:37.603407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:10:37.604898 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:10:37.673754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7205f-31aa-4152-9222-c2a8b26d16fd service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:10:37.678385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:10:37.679002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:10:37.679333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:10:37.679928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:10:37.690254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:10:37.692220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:10:37.705393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1148/2287] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:10:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:10:38.886315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:10:38.887964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:10:38.900250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659ce0-a10a-45ec-bbf2-caa1f50a7f9a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:10:38.903831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:10:38.904425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:10:38.904700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:10:38.904967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:10:38.915385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:10:38.916796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:10:38.927627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1140/2288] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:10:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:10:47.700481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=31 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:10:48.922683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:11:13.740803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:11:13.750510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026fece-8742-4a1b-b227-f184caed651a service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:11:13.754193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:11:13.754820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:11:13.755100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:11:13.755373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:11:13.757199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:11:13.766813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:11:13.768256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:11:13.776580 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1149/2289] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:11:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:11:17.813246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:11:17.823268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82120dc-78f5-4522-b6d8-f3f9a77139db service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:11:17.826803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018746d0-298e-4856-a570-61268797d6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:11:17.827395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018746d0-298e-4856-a570-61268797d6da service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:11:17.827677 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018746d0-298e-4856-a570-61268797d6da service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:11:17.827952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018746d0-298e-4856-a570-61268797d6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:11:17.894165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018746d0-298e-4856-a570-61268797d6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:11:17.895626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018746d0-298e-4856-a570-61268797d6da service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:11:17.917213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1141/2290] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:11:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:11:23.773109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:11:27.915138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=36 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:11:28.436101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:11:28.437629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:11:28.453256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfb7f-7c4c-4947-9551-753061aabfde service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:11:28.458811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:11:28.459413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:11:28.459697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:11:28.459973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:11:28.462246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:11:28.535654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:11:28.537232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:11:28.547073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1150/2291] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:11:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:11:30.700656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018746d0-298e-4856-a570-61268797d6da service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:11:30.710072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018746d0-298e-4856-a570-61268797d6da service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:11:30.714243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:11:30.714845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:11:30.715124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:11:30.715401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:11:30.724161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:11:30.725627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:11:30.735072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1142/2292] 10.222.0.82 () {60 vars in 1396 bytes} [Thu Aug 18 22:11:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:11:38.542960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=33 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:11:40.731024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:14:04.963486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:14:04.965644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:14:05.042264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb980-e426-462e-8d9c-73ce77fdb134 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:14:05.047929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:14:05.048545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:14:05.048829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:14:05.049120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:14:05.058161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:14:05.059809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:14:05.069957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1151/2293] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:14:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:14:15.065726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:14:55.848835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:14:55.880187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee129c29-ce9f-4797-a508-0c872bb05a9f service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:14:55.884186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:14:55.884791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:14:55.885073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:14:55.885346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:14:55.894472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:14:55.896135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:14:55.925740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1143/2294] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:14:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:14:57.600773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:14:57.639235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9100e-5d86-43dc-a97b-52ca688d7776 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:14:57.644824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:14:57.645424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:14:57.645734 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:14:57.646008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:14:57.653968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:14:57.655547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:14:57.664373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1152/2295] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:14:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:15:00.519744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:15:00.532453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ede7-5715-4e91-814f-42d481401a06 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:15:00.538700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:15:00.539346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:15:00.539625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:15:00.539897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:15:00.553503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:15:00.555159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:15:00.584346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1144/2296] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:15:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:15:07.660250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=18 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:15:10.581399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=35 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:15:36.967494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:15:36.976812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6247a-417e-4978-acaa-c388aa65b014 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:15:36.980371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:15:36.980968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:15:36.981249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:15:36.981523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:15:36.984045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:15:37.011673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:15:37.013118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:15:37.024248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1153/2297] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:15:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:15:38.208033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:15:38.218065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4095a-5885-493c-b7af-bd540159dd69 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:15:38.221678 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:15:38.222277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:15:38.222555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:15:38.222831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:15:38.224694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:15:38.239913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:15:38.241350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:15:38.253613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1145/2298] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:15:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:15:47.019439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=11 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:15:48.247625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:16:16.671857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:16:16.691132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8c8f-c96a-4456-8fa7-624ce54d5bd8 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:16:16.694798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:16:16.695404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:16:16.695683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:16:16.695957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:16:16.703460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:16:16.704900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:16:16.712789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1154/2299] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:16:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:16:18.122517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:16:18.124137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:16:18.137881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7f72b-03d9-462a-bf6e-a70ffdcaa1aa service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:16:18.143945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:16:18.144542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:16:18.144819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:16:18.145087 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:16:18.146880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:16:18.171593 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:16:18.173194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:16:18.182342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1146/2300] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:16:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:16:26.709350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:16:27.925956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:16:27.939666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee3fcb-77c8-4393-8687-5cdd4d161928 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:16:27.945226 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:16:27.946029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:16:27.946357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:16:27.946635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:16:27.955624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:16:27.957201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:16:27.966480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1155/2301] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:16:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:16:28.178262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=11 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:16:31.632829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:16:31.644082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbcde-f05d-4388-a977-a0169e206c8b service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:16:31.649875 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:16:31.650491 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:16:31.650773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:16:31.651050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:16:31.652904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:16:31.730205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:16:31.732449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:16:31.741487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1147/2302] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:16:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:16:33.055201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:16:33.076604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d87845-5365-4464-a579-8cae01c47e0e service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:16:33.088809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:16:33.090484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:16:33.091542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:16:33.094965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:16:33.098361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:16:33.117077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:16:33.119858 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf778c4a-e675-4028-97d9-2e3298549455 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:16:33.145838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1156/2303] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:16:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:16:41.737285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=33 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:16:43.135011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=12 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:17:07.262715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:07.275058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1b181-536e-43d3-bce4-a7421c15b8c4 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:07.282182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:07.283050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:07.283537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:07.284016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:07.286014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:07.299234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:07.301083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:07.312747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1148/2304] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:17:08.853934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:08.854446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:08.854574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:08.892827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] MFA Rules not processed for user `e280bab439a3446b993c721bc98c1c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:17:08.894066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:08.910915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1157/2305] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:17:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:17:09.220503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:09.229861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92607a39-0a22-422e-9e91-14fc06ee5724 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:09.234187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:09.234779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:09.235054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:09.235323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:09.237034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:09.247139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:09.248587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:09.256406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1149/2306] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:17:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:17:09.598542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:09.611497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6bfdf-c847-4b4f-b5a1-94be94155066 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:09.617387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:09.618131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:09.618599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:09.618880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:09.620814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:09.636332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:09.638517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529de1-7587-4865-ba33-cf60728cbc19 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:09.655015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1158/2307] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:17:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:17:17.349386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:17.363497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc113bee-ecef-4e3f-b80f-e657b8cf7617 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:17.368991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:17.369591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:17.369894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:17.370166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:17.371861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:17.384794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:17.386410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:17.397499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1150/2308] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:17:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:17:19.483191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:19.484057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:19.484333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:19.554217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:17:19.555592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:19.651865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1159/2309] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 22:17:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:17:19.745338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:19.747692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:19.874829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2795-0742-4d74-ad90-a6fcccbd854e service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:19.880218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:19.880854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:19.881138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:19.881414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:19.891132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:19.892775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27495923-b6a0-4c27-a5ba-fbd5e7219fb8 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:19.902574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1151/2310] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:17:20.656428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:17:20.668962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a88bb8-4195-4357-952b-71aef6ceb09d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:17:20.673009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:17:20.673878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:20.674323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:20.674732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:20.676708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:20.781211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:17:20.784981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:17:20.786679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:17:20.808896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1160/2311] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:17:23.865366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:17:23.865885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:17:23.866114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:17:23.907070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] MFA Rules not processed for user `7790a2ef47644d62863aafc890d03249`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:17:23.908587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:17:23.917410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1152/2312] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:17:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:17:30.805444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:17:33.903216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=65 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:19:09.967841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:19:09.969487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:19:10.039171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0bbc2-b153-4853-8db4-c2754ef922ee service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:19:10.044720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:19:10.045359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:19:10.045671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:19:10.045950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:19:10.047764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:19:10.059334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:19:10.061521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:19:10.071747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1161/2313] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:19:20.068017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:19:55.334435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:19:55.347311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e05c2-0f84-4b31-8ed3-cc5872ae7975 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:19:55.359222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:19:55.360334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:19:55.360608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:19:55.360879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:19:55.368925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:19:55.370533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:19:55.383540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1153/2314] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:19:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:20:05.375284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:20:33.630932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:20:33.632656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:20:33.700710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1baac-74d9-40e0-8418-205ed9be59be service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:20:33.704414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:20:33.705537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:20:33.705848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:20:33.706121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:20:33.707960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:20:33.780397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:20:33.781880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:20:33.792070 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1162/2315] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:20:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:20:33.840055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:20:33.849704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ddf34-eca9-4f8e-bd37-f388c7bdbfb1 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:20:33.853205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:20:33.853824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:20:33.854099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:20:33.854366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:20:33.856159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:20:33.923913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:20:33.925153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:20:33.934223 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1154/2316] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:20:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:20:37.359159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:20:37.368866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20774ec-e6bc-4a57-8abe-d1efce850856 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:20:37.372360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:20:37.372947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:20:37.373222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:20:37.373493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:20:37.439263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:20:37.441295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:20:37.452077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1163/2317] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:20:38.558553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:20:38.611728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea3a3b-bf00-466d-ab23-b6ab7b1791d3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:20:38.615316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:20:38.615913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:20:38.616187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:20:38.616452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:20:38.626323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:20:38.628235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f867087-0c51-432b-a6f3-2e6169fbc1bc service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:20:38.638747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1155/2318] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:20:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:20:47.447177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:20:48.634089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=59 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:21:18.654753 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:21:18.656306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:21:18.724640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0c66-91e1-46e5-b6e9-0d07e94d0d48 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:21:18.728370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:21:18.728968 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:21:18.729245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:21:18.729515 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:21:18.731362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:21:18.740839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:21:18.742375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:21:18.750411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1164/2319] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:21:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:21:28.746777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=31 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:22:33.263428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:33.263933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:33.264044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:33.305075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] MFA Rules not processed for user `a01cda302c5841cc937f219d7a06dc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:22:33.306945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:22:33.361316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1156/2320] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:22:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:22:33.661338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:33.662976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:22:33.673664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d221eb3-5084-4c2c-80fd-11bdd11442c6 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:33.677577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:33.678201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:33.678483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:33.679406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:33.681444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:22:33.691017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:33.692542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:33.700575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1165/2321] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:34.025290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:34.037200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd65381-3f94-4518-9cfa-38def75cf4fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:34.042925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:34.043533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:34.043808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:34.044079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:34.053631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:34.055516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:34.066150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1157/2322] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:34.167220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:34.177697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4cc26-57b4-427b-9027-a139995c8517 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:34.182044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:34.182823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:34.183175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:34.183696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:34.254050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:34.255492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:34.283006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1166/2323] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:34.334554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:34.344533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f3cb-9550-45e7-9d7a-afd38b7a8dd3 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:34.348848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:34.349583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:34.349887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:34.350194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:34.352215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:22:34.448251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:34.449703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:34.468867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1158/2324] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:34.696371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:34.705478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044491-4788-4dc5-9291-d89535566117 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:34.709137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:34.709770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:34.710080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:34.710352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:34.719025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:34.720440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:34.748834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1167/2325] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:36.431000 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:36.446841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac375-17ac-4bad-a399-cd6dc5ce7c21 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:36.453014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:36.453675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:36.453951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:36.454658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:36.464328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:36.465991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:36.478054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1159/2326] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:41.129533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:41.131480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:22:41.148262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91e2d2-03a6-476f-93d9-44fb930893ef service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:41.154017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:41.154640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:41.154925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:41.155249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:41.165072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:41.166713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:41.178278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1168/2327] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:44.679646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:22:44.692685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ddb34-da32-496f-9338-b8ac71c3ecd4 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:22:44.698427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:22:44.699430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:22:44.699430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:22:44.699633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:22:44.708747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:22:44.710368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:22:44.723117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1160/2328] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:22:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:22:51.173588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=61 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:22:54.718379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:24:16.956234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:24:16.957828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:24:17.031711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1178a-ca95-4860-9b65-aa0135c57d5a service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:24:17.036690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:24:17.037291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:24:17.037577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:24:17.037879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:24:17.046166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:24:17.047736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:24:17.057677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1169/2329] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:24:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:24:27.053334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:25:35.383926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:25:35.385498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:25:35.417028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec723f1-3155-4fe8-a2b3-54d8e9460b6b service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:25:35.421188 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:25:35.421820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:25:35.422099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:25:35.422371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:25:35.433808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:25:35.437172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:25:35.468127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1161/2330] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:25:35.551122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:25:35.570945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d2707-c2a1-474a-bcca-e21da481266f service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:25:35.574561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:25:35.575155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:25:35.575432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:25:35.575699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:25:35.583939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:25:35.585348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:25:35.594252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1170/2331] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:25:35.672885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:25:35.681465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591da9-e0b1-47dc-8104-0258fb3f04e3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:25:35.685956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:25:35.686550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:25:35.686826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:25:35.687096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:25:35.688943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:25:35.709522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:25:35.710979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:25:35.729723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1162/2332] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:25:45.590138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=11 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:25:45.727113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=36 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:26:23.258267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:26:23.276886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88d55c-5249-4999-ac32-ea76dbc7442d service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:26:23.281454 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:26:23.282072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:26:23.282351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:26:23.282626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:26:23.289897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:26:23.291320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:26:23.299933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1171/2333] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:26:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:26:33.296357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:27:35.847114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:27:35.855663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470a76-cc9f-420d-bf09-ae1464e6d161 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:27:35.859784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:27:35.860378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:27:35.860656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:27:35.860928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:27:35.862762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:27:35.886435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:27:35.887901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:27:35.903175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1163/2334] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:27:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:27:35.983438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:27:35.992426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47aa21-8403-4260-92f8-68b8c187be04 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:27:35.995912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:27:35.996498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:27:35.996774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:27:35.997711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:27:36.007129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:27:36.008547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:27:36.017278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1172/2335] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:27:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:27:45.898889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=11 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:27:46.013311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:28:35.566636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:28:35.575034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614329a9-e142-4bd9-ac7b-988784d6c2f8 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:28:35.579243 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:28:35.579831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:28:35.580106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:28:35.580374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:28:35.582219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:28:35.651481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:28:35.652848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:28:35.661847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1164/2336] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:28:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:28:45.657690 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=33 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:23.945894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:24.011983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2797-5c8a-49db-af1c-25b41ad09b50 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:24.017398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:24.018035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:24.018311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:24.018582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:24.020423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:24.032161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:24.033752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:24.044343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1173/2337] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:34.040153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:35.272732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:35.286122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516f628-d7e4-40e3-863b-ee0e932ee09c service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:35.297046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:35.297800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:35.301763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:35.302313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:35.304880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:35.355124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:35.357904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:35.365347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:35.376356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09a190-9689-4eed-b272-e453f999ebd2 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:35.380991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:35.381940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:35.382298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:35.382587 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:35.383415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1165/2338] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:35.396269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:35.425574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:35.427703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:35.444588 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1174/2339] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:37.502149 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:37.532324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efee9-4e29-4775-9203-98660e7952e3 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:37.538329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:37.539397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:37.539847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:37.540388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:37.546553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:37.670391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:37.672552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:37.703115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1166/2340] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:37.905270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:37.923525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef91717-af2b-46e3-8f83-cda2a255a23e service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:37.938574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:37.939205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:37.939487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:37.939763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:37.943485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:37.976044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:37.977801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:38.015435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1175/2341] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:38.039110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:38.040880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:38.076686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ee97e-7a79-4e37-8b55-55ab0e2f0502 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:38.089781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:38.090453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:38.091062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:38.091483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:38.096783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:38.126702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:38.128350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:38.157024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1167/2342] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:42.386827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:42.389136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:42.417507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e735c-dbe4-4521-858f-6280fa7cd803 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:42.425864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:42.426473 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:42.427108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:42.427381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:42.429247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:42.482376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:42.484011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:42.525006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1176/2343] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:42.626983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:42.628408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:42.650888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fff647-efb3-476b-bc34-cf03c3b8f3c6 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:42.657011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:42.667376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:42.672716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:42.673319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] PATH_INFO: `/v3/users/a01cda302c5841cc937f219d7a06dc56` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:42.676548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01cda302c5841cc937f219d7a06dc56)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:42.681668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:42.794974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01cda302c5841cc937f219d7a06dc56'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:42.795959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01cda302c5841cc937f219d7a06dc56'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:42.796926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01cda302c5841cc937f219d7a06dc56'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:42.817166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01cda302c5841cc937f219d7a06dc56'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:42.820435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:42.829175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2c316915-62ba-46d7-b298-666ca4fdf942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01cda302c5841cc937f219d7a06dc56'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:42.852035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1168/2344] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:42 2022] DELETE /identity/v3/users/a01cda302c5841cc937f219d7a06dc56 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:43.336622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.347360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd64d1-3f52-4bcb-9d8d-15d18b7b3fe5 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.415927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.416765 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.416867 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.417097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] PATH_INFO: `/v3/projects/1c52341f519e47f59defbd74b623efac` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.418266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c52341f519e47f59defbd74b623efac)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.421012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.472425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c52341f519e47f59defbd74b623efac', 'request_id': 'req-b0c07bbb-2d58-4bd2-a92e-58df673d6451'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:43.483345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c52341f519e47f59defbd74b623efac', 'request_id': 'req-b0c07bbb-2d58-4bd2-a92e-58df673d6451'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:43.490088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c52341f519e47f59defbd74b623efac', 'request_id': 'req-b0c07bbb-2d58-4bd2-a92e-58df673d6451'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:43.503801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] The token cache is being invalidate because project 1c52341f519e47f59defbd74b623efac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:43.504091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:43.504747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0c07bbb-2d58-4bd2-a92e-58df673d6451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:43.506280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1177/2345] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:43 2022] DELETE /identity/v3/projects/1c52341f519e47f59defbd74b623efac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:43.512776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.513132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.513743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.579722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:43.581056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:43.655105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1169/2346] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:43.658864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.659611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.659611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.682980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:43.684209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:43.696585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1178/2347] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:43.701525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.712206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c94e-c3f3-4e59-a452-b677b822e9a0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.717699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.718324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.718604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.718891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.719538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:43.719841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.721451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.734948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1170/2348] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:43.739140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.749700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ed34f-7af9-4f08-b661-7df14d4ea816 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.754929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.755547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.755955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.756247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.756948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.759109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.809665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1179/2349] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:43 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:43.812280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.825871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d063a4-9fa0-4532-98b3-2f021970bcfd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.891684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.892834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.892834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.892834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.894135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.896367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.923336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1171/2350] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:43 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:29:43.927635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.938310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f6037-d393-4a22-b433-4f088352cfa2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.942973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.943582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.943861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.944136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.944813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.946439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.957812 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1180/2351] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:43.961976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:43.973880 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bf3b9-b38a-49b8-8ffc-ced6e5581851 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:43.978575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:43.979176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:43.979525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:43.979722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:43.980359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:43.982050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ec845-6d5c-4219-81a7-27f87459fc3a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:43.994128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1172/2352] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:43.998730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:44.004716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:44.011507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842c9a-d019-4e0c-8ab7-d433ce80e6a9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:44.018024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:44.018807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:44.019117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:44.019393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] PATH_INFO: `/v3/projects/e2642252d638481098b8fc1d814011be/users/9cd2c566e1344155b62da300dd8b510d/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:44.035460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2642252d638481098b8fc1d814011be, user_id=9cd2c566e1344155b62da300dd8b510d, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:44.038158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:44.053244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1181/2353] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:43 2022] PUT /identity/v3/projects/e2642252d638481098b8fc1d814011be/users/9cd2c566e1344155b62da300dd8b510d/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:29:44.064031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:44.064310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:44.064581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:44.094200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] MFA Rules not processed for user `9cd2c566e1344155b62da300dd8b510d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:44.095414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:44.103543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:44.168684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1173/2354] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:44.179547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:44.181328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:44.261030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690ed0-cc83-4043-9ab2-9a5a6e98218c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:44.267610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:44.268310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:44.268666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:44.269012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:44.284920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:44.287573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:44.309950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1182/2355] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:46.262604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:46.264185 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:46.346205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72986c2f-2c58-4e7a-beb4-0d4c17749052 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:46.350392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:46.351232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:46.351642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:46.352101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:46.354583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:46.365130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:46.438285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefe763-971a-4bc8-91c0-428d40c23c84 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:46.444271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:46.444471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:46.445311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:46.445768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:46.446095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab8133-348d-47ce-86ae-bda37f3684d2 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:46.446409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] PATH_INFO: `/v3/users/9cd2c566e1344155b62da300dd8b510d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:46.447727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd2c566e1344155b62da300dd8b510d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:46.449841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:46.457681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1174/2356] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:46.507046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd2c566e1344155b62da300dd8b510d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:46.518193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd2c566e1344155b62da300dd8b510d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:46.528707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:46.535763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd2c566e1344155b62da300dd8b510d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:46.547617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd2c566e1344155b62da300dd8b510d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:46.548330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-06dcf31d-08a6-4c63-9987-21e7ec909b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd2c566e1344155b62da300dd8b510d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:46.556365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1183/2357] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:46 2022] DELETE /identity/v3/users/9cd2c566e1344155b62da300dd8b510d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:46.561763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:46.562054 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:46.562832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:46.585997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:46.587309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:46.590434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:46.591435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:46.591435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:46.622864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] MFA Rules not processed for user `7790a2ef47644d62863aafc890d03249`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:46.629155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:46.654870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:46.670059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1175/2358] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:46.681492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:46.683295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:46.749681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1184/2359] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:46.770964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969973d-4962-40af-8bb5-f4770fac8131 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:46.777236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:46.778703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:46.779051 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:46.779330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:46.789912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:46.791399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:46.809008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1176/2360] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:47.064831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.074612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a388c57-92a2-4a12-871b-82f90d37fd21 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:47.079031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:47.079710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.080060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.080403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.082212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.095720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:47.097321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:47.108047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1185/2361] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:47.463357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.465041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.466842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.469249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.481260 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b4c37-57b5-41a2-b277-d27cefa4f74d service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:47.482166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83e4b-a10a-4dd5-9dbf-4f53600156a7 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:47.486649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:47.488220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.488641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:47.488942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.489278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.489679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.489784 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.490060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] PATH_INFO: `/v3/projects/e2642252d638481098b8fc1d814011be` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.491962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2642252d638481098b8fc1d814011be)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:47.492932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.494725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:47.520578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:47.522769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab590bd3-fc89-4a9f-b48f-53fcd7219583 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:47.536418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1186/2362] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:47.570514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2642252d638481098b8fc1d814011be', 'request_id': 'req-80abc594-52fa-4907-9279-f4b033a481e1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:47.580323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2642252d638481098b8fc1d814011be', 'request_id': 'req-80abc594-52fa-4907-9279-f4b033a481e1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:47.591754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2642252d638481098b8fc1d814011be', 'request_id': 'req-80abc594-52fa-4907-9279-f4b033a481e1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:47.605053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] The token cache is being invalidate because project e2642252d638481098b8fc1d814011be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:47.605720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:47.606589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-80abc594-52fa-4907-9279-f4b033a481e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:47.612819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1177/2363] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:47 2022] DELETE /identity/v3/projects/e2642252d638481098b8fc1d814011be => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:47.613877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.614166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.614446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.639016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:47.640409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.722511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1187/2364] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:47.731056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.731878 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.732168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.752769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:47.754033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:47.762662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1178/2365] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:47.767054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.779731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb3e4-4dc4-4ce4-8d29-6836575161ce None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:47.784687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:47.785310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.785585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.785888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.786524 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:47.786826 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:47.788423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:47.809190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1188/2366] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:47.812094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.822526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ff8c9-e2fa-486e-a733-5b1e4fcd0a53 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:47.831739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:47.832221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:47.832772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:47.832881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:47.833495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:47.847439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:47.923031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1179/2367] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:47 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:47.931168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:47.946296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d249-ad26-4ccf-8a10-16c27b383326 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:48.015838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:48.056639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:48.057310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.057951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.057951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.058686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:48.063224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:48.106980 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1189/2368] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:47 2022] POST /identity/v3/users => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:29:48.110753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:48.122585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18fc9a-a57d-4ede-93f8-d58b9416b9d6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:48.127372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:48.127992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.128251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.128533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.129153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:48.130793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:48.143670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1180/2369] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:29:48.148509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:48.159789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069967-e4ef-4b72-be7a-a7a25553d8e2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:48.180248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:48.180891 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.181167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.181437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.182137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:48.184250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa587200-0834-4d6c-a43e-8d7ba05f46db admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:48.204424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1190/2370] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:48.207629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:48.235276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f55bf-d046-4bf1-92ea-0ae0197e3f2f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:48.242812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:48.243671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.244018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.244360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] PATH_INFO: `/v3/projects/ba5bcf9ddb9140f1b91a041b76f9a3fb/users/5c195d8ab41f4cf79255d79a27aa8e51/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.276918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5bcf9ddb9140f1b91a041b76f9a3fb, user_id=5c195d8ab41f4cf79255d79a27aa8e51, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:48.281634 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:48.312221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1181/2371] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:48 2022] PUT /identity/v3/projects/ba5bcf9ddb9140f1b91a041b76f9a3fb/users/5c195d8ab41f4cf79255d79a27aa8e51/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:29:48.317400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.317727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.318008 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.344693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:48.346725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:48.448111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1191/2372] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:48.460225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:48.462163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:48.505365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:48.545443 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e1239-ff7f-4a49-8b81-bdfe52371cbd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:48.553392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:48.555244 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:48.555775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:48.557570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:48.573351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:48.575479 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a38a0-97b2-4549-a1bc-a6bd322de429 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:48.586342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1182/2373] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:49.108978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:49.110494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:49.210984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84c6c-4014-43bd-8d7f-ffc18fe408cd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:49.219957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:49.220850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:49.222184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:49.223750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:49.227509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:49.251784 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:49.254165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:49.277078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1192/2374] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:50.243181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:50.243724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:50.243809 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:50.319076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] MFA Rules not processed for user `5c195d8ab41f4cf79255d79a27aa8e51`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:50.320233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:50.546540 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1183/2375] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:50.547583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:50.569689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c4b7-23ef-4572-8cc2-4a7aa774076f service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:50.583360 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:50.585177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:50.586949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:50.587452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:50.588395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:50.594389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:50.660683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1193/2376] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:50 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:50.665239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:50.684090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7341-767c-4c97-873e-5dac2275c0b4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:50.771463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:50.772035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:50.772318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:50.773473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:50.774175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:50.776983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:50.815207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1184/2377] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:50 2022] POST /identity/v3/users => generated 308 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:50.819093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:50.830406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655428c-070a-4c13-849c-21fcf391c211 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:50.837485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:50.838796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:50.839086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:50.840374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:50.841089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:50.850391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:50.869106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1194/2378] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:50.872898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:50.889451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfcbc1-26fb-460f-aa88-c28bad7b8322 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:50.894981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:50.895660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:50.896176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:50.896750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] PATH_INFO: `/v3/projects/8afae0ccf5224113bd0508436c228734/users/0f83152c8753499696365f9d03f84fb1/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:50.917554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afae0ccf5224113bd0508436c228734, user_id=0f83152c8753499696365f9d03f84fb1, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:50.923963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:50.949019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1185/2379] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:50 2022] PUT /identity/v3/projects/8afae0ccf5224113bd0508436c228734/users/0f83152c8753499696365f9d03f84fb1/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:50.951077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:50.987145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa93a2-0529-4873-9997-5a1e8d448594 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:51.014459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:51.105528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:51.106971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:51.107265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:51.108745 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:51.113215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:51.115963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:51.136084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1195/2380] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:51.140008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:51.152559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65c2b-b46e-4b9c-9632-98dac9fa64d6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:51.158372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:51.159220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:51.159604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:51.160101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] PATH_INFO: `/v3/projects/8afae0ccf5224113bd0508436c228734/users/0f83152c8753499696365f9d03f84fb1/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:51.163668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afae0ccf5224113bd0508436c228734, user_id=0f83152c8753499696365f9d03f84fb1, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:51.173207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1460e-38b2-463d-8cfe-df1b4a3744c7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:51.191570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1186/2381] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:51 2022] PUT /identity/v3/projects/8afae0ccf5224113bd0508436c228734/users/0f83152c8753499696365f9d03f84fb1/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:51.992257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:51.994663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.015564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cab91-20f5-4d6f-88cc-7816823c717d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.108169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.109110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.109852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.110262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] PATH_INFO: `/v3/users/e280bab439a3446b993c721bc98c1c6e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.111488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e280bab439a3446b993c721bc98c1c6e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.113874 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.170145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.170929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.171423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.185458 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e280bab439a3446b993c721bc98c1c6e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.193514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:52.194429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e280bab439a3446b993c721bc98c1c6e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.217801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e280bab439a3446b993c721bc98c1c6e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.218395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] MFA Rules not processed for user `0f83152c8753499696365f9d03f84fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:52.219718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.230947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e280bab439a3446b993c721bc98c1c6e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.233447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e280bab439a3446b993c721bc98c1c6e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.245182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1196/2382] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:51 2022] DELETE /identity/v3/users/e280bab439a3446b993c721bc98c1c6e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:52.276869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.278336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.322584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1187/2383] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:52.346107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:52.401289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f746e-dae6-4451-a3a9-24b876340967 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.409434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.410797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.411637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.412123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] PATH_INFO: `/v3/users/42072422001c4f1684f68baa25517a9e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.414685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42072422001c4f1684f68baa25517a9e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.418305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.480565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.482928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.501586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11edf2-602c-4102-96bd-cfb2abd369cc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.504806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42072422001c4f1684f68baa25517a9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.507132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.507944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.508226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.508500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] PATH_INFO: `/v3/projects/efb811ce4fca47728e61489adbf97434` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.522688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb811ce4fca47728e61489adbf97434)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.524963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.534797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42072422001c4f1684f68baa25517a9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.551175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42072422001c4f1684f68baa25517a9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.559179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42072422001c4f1684f68baa25517a9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.561263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-08330464-0bb7-43a3-ac96-de678052ea6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42072422001c4f1684f68baa25517a9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.581563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb811ce4fca47728e61489adbf97434', 'request_id': 'req-35530d75-3ba6-4bcd-b96a-dec5dd37c775'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.588341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb811ce4fca47728e61489adbf97434', 'request_id': 'req-35530d75-3ba6-4bcd-b96a-dec5dd37c775'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.592174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1197/2384] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:52 2022] DELETE /identity/v3/users/42072422001c4f1684f68baa25517a9e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:52.604553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb811ce4fca47728e61489adbf97434', 'request_id': 'req-35530d75-3ba6-4bcd-b96a-dec5dd37c775'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.625832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] The token cache is being invalidate because project efb811ce4fca47728e61489adbf97434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:52.626706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.627080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.629724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1188/2385] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:52 2022] DELETE /identity/v3/projects/efb811ce4fca47728e61489adbf97434 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:29:52.635755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69543b-2d66-4314-9fe0-4ffc44b78469 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.636049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69543b-2d66-4314-9fe0-4ffc44b78469 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.636605 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69543b-2d66-4314-9fe0-4ffc44b78469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.667302 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8e69543b-2d66-4314-9fe0-4ffc44b78469 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:52.670186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8e69543b-2d66-4314-9fe0-4ffc44b78469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.736699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.738335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.767976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1198/2386] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:52.775657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.775956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.776235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.802170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:52.803699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.815948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1199/2387] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:52.820838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.826649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35530d75-3ba6-4bcd-b96a-dec5dd37c775 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.844271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452d4e8-04d3-4321-80d7-a0d023f342f4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.850265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.850942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.851173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.851452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.852185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:52.852407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.854103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.864151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:52.872694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1200/2388] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:29:52.873664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.875299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.891744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315157-ccd6-4ac5-95b7-cbfa8f0fdd84 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.898726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.899938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.900197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.900477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] PATH_INFO: `/v3/projects/9894625c04a341e3a82584481410cd35` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.902230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9894625c04a341e3a82584481410cd35)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.904504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.911648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:52.912285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:52.912595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:52.912869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:52.914795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:52.930776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:52.932330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:52.943554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1189/2389] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:52.944456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:52.945435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:52.955687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaaab9-0f6c-414e-9d95-262349709ca4 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:52.967119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9894625c04a341e3a82584481410cd35', 'request_id': 'req-82ac9302-5e02-427d-86a1-4f4fc65489a4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.979159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9894625c04a341e3a82584481410cd35', 'request_id': 'req-82ac9302-5e02-427d-86a1-4f4fc65489a4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:52.987863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9894625c04a341e3a82584481410cd35', 'request_id': 'req-82ac9302-5e02-427d-86a1-4f4fc65489a4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:53.001863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] The token cache is being invalidate because project 9894625c04a341e3a82584481410cd35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:53.002501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:53.002957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82ac9302-5e02-427d-86a1-4f4fc65489a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:53.004751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1201/2390] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:52 2022] DELETE /identity/v3/projects/9894625c04a341e3a82584481410cd35 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:53.011377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c5fa-d140-4b2c-90f3-f70c31d3ba8e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.013154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c5fa-d140-4b2c-90f3-f70c31d3ba8e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.013579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c5fa-d140-4b2c-90f3-f70c31d3ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.035698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.036477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.036477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.036675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.037662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.039567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.056850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2a70c5fa-d140-4b2c-90f3-f70c31d3ba8e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:53.058318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2a70c5fa-d140-4b2c-90f3-f70c31d3ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.070553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1202/2391] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:53.082551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.083164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.084222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.095488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1190/2392] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:52 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:53.096544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.099718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.128624 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:53.131444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.204100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d28e0-2837-43f4-a9c8-b1c46a1c8bbd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.210180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.211090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.211595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.212105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.212933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.215294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.242115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1203/2393] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:53.247249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.256694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc60cd-c7ca-4227-ac1c-c23e381d99a3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.263351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.263982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.264356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.264642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.265300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:53.265612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.267269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.275465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1191/2394] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:53.277673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.282319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1204/2395] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:53.287512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.300573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776a14c-d716-466f-a8df-543d53056735 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.307160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b54e-7b75-471c-b64a-2e60d71a0800 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.315613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.316259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.316557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.316810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.317561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.317726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.318675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.319178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.319649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.320502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.322465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.323752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.339768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1192/2396] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:53.342596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.355939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437995b-61f2-46f0-92f5-0f78ff9bcc95 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.363707 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.364814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.365351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.365879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.366772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.368598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.382644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1193/2397] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:53.384379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1205/2398] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:29:53.386841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.390539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.403103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8f3-5d16-4faa-9439-e1f3a9dc7d78 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.423279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:53.478513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66b3e-e894-44b2-853a-9473c7a5b634 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.483978 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.484792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.485183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.485566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.486342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.488411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.490622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.492316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.492316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.492316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] PATH_INFO: `/v3/projects/0a626e05e86b49b09f708bff1307eddd/users/275fdd34fad04902a38ea3ff88501065/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.516175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a626e05e86b49b09f708bff1307eddd, user_id=275fdd34fad04902a38ea3ff88501065, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.520893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1206/2399] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/users => generated 282 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:29:53.521056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3112-284d-4c1c-a416-0633cc2d8265 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.522469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.531790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:53.536729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009545c3-2dfd-45fa-96f3-8085900d5e26 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.540662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1194/2400] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:53 2022] PUT /identity/v3/projects/0a626e05e86b49b09f708bff1307eddd/users/275fdd34fad04902a38ea3ff88501065/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:53.550927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.551346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.551632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.607343 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] MFA Rules not processed for user `275fdd34fad04902a38ea3ff88501065`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:53.616078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.646054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.646695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.647172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.647288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.647887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.649547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.663274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1207/2401] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:29:53.667397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.682500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34363373-60ab-4072-b57a-78b8ca38ff43 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.690096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.690728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.691038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.691280 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.692154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.695171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.714147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1208/2402] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:53.716554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.730597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592cb1-435f-4ea0-8017-67856daa45fa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.738351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:53.739237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.740830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.741295 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] PATH_INFO: `/v3/projects/280177513f744247a5c5aa1e2021dc68/users/d4f808f6f19243e3823ec53c12bc0af9/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.771049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=280177513f744247a5c5aa1e2021dc68, user_id=d4f808f6f19243e3823ec53c12bc0af9, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:53.774890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348453e-0b39-4293-9e38-235157d498e0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:53.783990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1195/2403] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:53.795854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:53.798900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.802101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1209/2404] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:53 2022] PUT /identity/v3/projects/280177513f744247a5c5aa1e2021dc68/users/d4f808f6f19243e3823ec53c12bc0af9/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:53.821083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:53.821619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:53.821997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:53.872304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] MFA Rules not processed for user `d4f808f6f19243e3823ec53c12bc0af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:53.873679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:53.956137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1099d-bfa7-4260-ab82-63a87b0ce163 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:53.981559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1210/2405] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:53.996062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.000352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.008004 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:54.063490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.064230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.064601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.064942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.094845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:54.116992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2786f5-0a27-4143-bbdd-3ca018c52841 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.125339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.126506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.127354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.128136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] PATH_INFO: `/v3/users/6a40fd568da34b26bd93b8ff0ea92812` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.129986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a40fd568da34b26bd93b8ff0ea92812)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.132238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.138704 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.140742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.151028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1196/2406] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:54.151937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.165069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcd5ca-9044-4b3f-9e50-28dc468fd2cf service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.171826 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.172541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.172822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.173154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.173828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.175741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842f285-e8f6-4c12-912a-58f37c1e019f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.200662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a40fd568da34b26bd93b8ff0ea92812'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.210148 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a40fd568da34b26bd93b8ff0ea92812'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.222478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a40fd568da34b26bd93b8ff0ea92812'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.242287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a40fd568da34b26bd93b8ff0ea92812'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.243739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a40fd568da34b26bd93b8ff0ea92812'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.253424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1211/2407] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:53 2022] DELETE /identity/v3/users/6a40fd568da34b26bd93b8ff0ea92812 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:54.266791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1197/2408] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:54 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:54.269264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.271118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.372978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfaac4-16a3-423d-ac98-8ead36738cf5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.373297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfaac4-16a3-423d-ac98-8ead36738cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.373747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfaac4-16a3-423d-ac98-8ead36738cf5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.376676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1198/2409] 10.222.0.82 () {54 vars in 921 bytes} [Thu Aug 18 22:29:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 22:29:54.384292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.384292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.384292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.393195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81c1ba-6158-4454-b6b2-c438ef0f6c37 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.503478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.504256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.505109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.505395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.506065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.508168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.524175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] MFA Rules not processed for user `6ebbb44da1cc45218ce9bb43f11e6715`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:54.525752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.534706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1212/2410] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:54 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:54.538709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.541130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.553615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b172b-c7b0-429d-b393-460ae6ab9612 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.559457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.560096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.560371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.560893 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] PATH_INFO: `/v3/users/275fdd34fad04902a38ea3ff88501065` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.562844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=275fdd34fad04902a38ea3ff88501065)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.564743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.570626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:54.612777 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1199/2411] 10.222.0.82 () {58 vars in 1018 bytes} [Thu Aug 18 22:29:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:54.613951 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.617084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.634211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275fdd34fad04902a38ea3ff88501065'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.639045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883f187-99d2-4cd5-be0e-85c45b04e6b5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.642663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275fdd34fad04902a38ea3ff88501065'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.655980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.656359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.656576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.656921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] PATH_INFO: `/v3/users/5c195d8ab41f4cf79255d79a27aa8e51` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.658280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c195d8ab41f4cf79255d79a27aa8e51)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.660265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.661663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275fdd34fad04902a38ea3ff88501065'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.671247 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275fdd34fad04902a38ea3ff88501065'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.672799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275fdd34fad04902a38ea3ff88501065'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.676073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:54.680545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1213/2412] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:54 2022] DELETE /identity/v3/users/275fdd34fad04902a38ea3ff88501065 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:54.685521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.693837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.730252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c195d8ab41f4cf79255d79a27aa8e51'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.731084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c195d8ab41f4cf79255d79a27aa8e51'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.731926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c195d8ab41f4cf79255d79a27aa8e51'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.748277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c195d8ab41f4cf79255d79a27aa8e51'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.756545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c195d8ab41f4cf79255d79a27aa8e51'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:54.773258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1200/2413] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:54 2022] DELETE /identity/v3/users/5c195d8ab41f4cf79255d79a27aa8e51 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:54.777495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.779326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.798135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b2e6a-a1c5-49d2-ae62-f63bf03e06ae admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.880697 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.881632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.881934 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.882283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.883268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.886257 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.899119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1214/2414] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:54.906162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:54.908092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.911653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65190d-11a0-4bf0-aa70-0e8b5e90e602 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.923568 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.925290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.925799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.927326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.928940 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727713ab-615c-4302-b77e-f9d8486dd6a4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:54.932978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:54.935956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:54.937832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:54.938075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:54.938418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] PATH_INFO: `/v3/projects/3b157ed00f1548f1af769106e03cf9b5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:54.939855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b157ed00f1548f1af769106e03cf9b5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.942056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.964565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:54.966997 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac00ca-1ddc-42cc-b6ca-38829e7747b7 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:54.978642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:55.032602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b157ed00f1548f1af769106e03cf9b5', 'request_id': 'req-60865624-363a-490a-9ca0-086891332e63'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.034793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1201/2415] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:55.036269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc8474-2ee9-4dc5-9b62-ccec5713da82 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.036571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc8474-2ee9-4dc5-9b62-ccec5713da82 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.036848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc8474-2ee9-4dc5-9b62-ccec5713da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.048130 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b157ed00f1548f1af769106e03cf9b5', 'request_id': 'req-60865624-363a-490a-9ca0-086891332e63'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.062036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b157ed00f1548f1af769106e03cf9b5', 'request_id': 'req-60865624-363a-490a-9ca0-086891332e63'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.063651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-defc8474-2ee9-4dc5-9b62-ccec5713da82 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:55.065783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-defc8474-2ee9-4dc5-9b62-ccec5713da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.096278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] The token cache is being invalidate because project 3b157ed00f1548f1af769106e03cf9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:55.098103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.112346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.114945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1215/2416] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:54 2022] DELETE /identity/v3/projects/3b157ed00f1548f1af769106e03cf9b5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:55.116864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.118907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.216927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1202/2417] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:55.222638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.222945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.223221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.227723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60865624-363a-490a-9ca0-086891332e63 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.237169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.238069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.238444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.238787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] PATH_INFO: `/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f/users/442e2ecb7bc84a4bb998cde9e95931cd/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.251346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:55.252818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.258716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4388ae1d9f40bc9d0754e1b8a62f6f, user_id=442e2ecb7bc84a4bb998cde9e95931cd, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.261879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.263445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1203/2418] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:55.264284 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.265701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.278216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed4ba3-466a-451a-941f-71ea1383c3e7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.283569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.284208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.284495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.284753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] PATH_INFO: `/v3/projects/ba5bcf9ddb9140f1b91a041b76f9a3fb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.285989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5bcf9ddb9140f1b91a041b76f9a3fb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.287964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.294123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1216/2419] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:55 2022] PUT /identity/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f/users/442e2ecb7bc84a4bb998cde9e95931cd/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:55.295883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.297821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:55.299039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.354286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5bcf9ddb9140f1b91a041b76f9a3fb', 'request_id': 'req-5c90a422-ec4c-484e-aa19-512eba761634'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.362783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5bcf9ddb9140f1b91a041b76f9a3fb', 'request_id': 'req-5c90a422-ec4c-484e-aa19-512eba761634'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.371446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5bcf9ddb9140f1b91a041b76f9a3fb', 'request_id': 'req-5c90a422-ec4c-484e-aa19-512eba761634'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.387009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] The token cache is being invalidate because project ba5bcf9ddb9140f1b91a041b76f9a3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:55.387220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.388024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5c90a422-ec4c-484e-aa19-512eba761634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.388922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627070-32f1-417a-972e-d585eea5f4a1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.393171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1204/2420] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:55 2022] DELETE /identity/v3/projects/ba5bcf9ddb9140f1b91a041b76f9a3fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:55.395642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.395937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.396359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.419398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:55.420713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.451703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:55.459742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.460747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.461101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.462325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.463945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.499298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1205/2421] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:55.500506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.505060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.518044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3dab6-dd41-4021-9a85-89c279d26c32 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.524665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.525589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.525885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.526408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.527200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.528918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.540097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1206/2422] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:55.540928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.546711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.546998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.548594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.561007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d57ab-b8e1-4733-be63-3170fc9e0b7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.561177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1217/2423] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:55.567378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.568039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.568332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.568785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] PATH_INFO: `/v3/users/0f83152c8753499696365f9d03f84fb1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.570484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f83152c8753499696365f9d03f84fb1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.573080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.573660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.594716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d2a9a-244a-429e-8623-5173255818eb service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.601925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.603638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.604760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.604760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.605446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:55.605862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.608981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.624718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1218/2424] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:55.625518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.641465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce9f-bd37-4440-a1e5-f073d2810444 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.648313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.649569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.649933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.650335 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] PATH_INFO: `/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f/users/442e2ecb7bc84a4bb998cde9e95931cd/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.654115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4388ae1d9f40bc9d0754e1b8a62f6f, user_id=442e2ecb7bc84a4bb998cde9e95931cd, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.658241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.676052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1219/2425] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:55 2022] PUT /identity/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f/users/442e2ecb7bc84a4bb998cde9e95931cd/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:55.676254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f83152c8753499696365f9d03f84fb1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.682567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.685047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f83152c8753499696365f9d03f84fb1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.686257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f83152c8753499696365f9d03f84fb1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.720923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f83152c8753499696365f9d03f84fb1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.729962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c515bf0-d536-41ab-945a-6f24cc38f85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f83152c8753499696365f9d03f84fb1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:55.747940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:55.757570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1207/2426] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:55 2022] DELETE /identity/v3/users/0f83152c8753499696365f9d03f84fb1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:55.759726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.760153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.760661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.805161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] MFA Rules not processed for user `442e2ecb7bc84a4bb998cde9e95931cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:55.809274 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.848875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3c30-5035-4b2d-952e-13bfaba0ff0f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:55.926816 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:55.927598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:55.927995 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:55.928368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:55.929219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:55.935949 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:55.936860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1208/2427] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:55.947078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.948841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:55.978911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1220/2428] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:55 2022] POST /identity/v3/projects => generated 399 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:55.980184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:55.983726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.002284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:56.060490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1ec4-5db8-40ce-8bd0-54246bfe433a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.092985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee26a2-7a84-4836-a00c-10b72842731a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.098586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.099354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.099701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.100048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] PATH_INFO: `/v3/projects/0a626e05e86b49b09f708bff1307eddd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.101241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a626e05e86b49b09f708bff1307eddd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.103695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.116588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.117209 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.117489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.117886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.162231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a626e05e86b49b09f708bff1307eddd', 'request_id': 'req-876787ab-4c68-4fe1-a691-5130d9bee65c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.171418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a626e05e86b49b09f708bff1307eddd', 'request_id': 'req-876787ab-4c68-4fe1-a691-5130d9bee65c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.178447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a626e05e86b49b09f708bff1307eddd', 'request_id': 'req-876787ab-4c68-4fe1-a691-5130d9bee65c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.187156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:56.193716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] The token cache is being invalidate because project 0a626e05e86b49b09f708bff1307eddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:56.194009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.194665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.199693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1221/2429] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:55 2022] DELETE /identity/v3/projects/0a626e05e86b49b09f708bff1307eddd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:56.199693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.200462 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.200559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.201012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409954ff-5c57-4b79-ab4c-209fac47fb6f service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.212555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1209/2430] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:56.213845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.214127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.214395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.235941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:56.237154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.303925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876787ab-4c68-4fe1-a691-5130d9bee65c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.311167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.312156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.312453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.312721 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.313362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.315332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.339375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1210/2431] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:56.340133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.342290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.352576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1222/2432] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:56.353930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.355496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216991a-51b6-4def-9edd-97dd6dccdd53 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.358074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.361061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.361751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.362029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.362333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] PATH_INFO: `/v3/projects/8afae0ccf5224113bd0508436c228734` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.364134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afae0ccf5224113bd0508436c228734)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.366367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.389103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:56.430794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afae0ccf5224113bd0508436c228734', 'request_id': 'req-2be4f14d-fec9-4ca7-9dc8-868813498d84'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.433457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4fd30-8806-4822-a561-c7495c1ba103 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.439099 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afae0ccf5224113bd0508436c228734', 'request_id': 'req-2be4f14d-fec9-4ca7-9dc8-868813498d84'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.449574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afae0ccf5224113bd0508436c228734', 'request_id': 'req-2be4f14d-fec9-4ca7-9dc8-868813498d84'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.465955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] The token cache is being invalidate because project 8afae0ccf5224113bd0508436c228734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:29:56.466533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.468542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2be4f14d-fec9-4ca7-9dc8-868813498d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:29:56.471060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1211/2433] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:29:56 2022] DELETE /identity/v3/projects/8afae0ccf5224113bd0508436c228734 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:56.472408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.472688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.472963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.495113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.495822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.496324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.496614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.499259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:56.499406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.500508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.570600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1212/2434] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:56.571594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.573233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.576833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.578356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8da7a-168b-40ca-b1a8-2556b0e9356b service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.586189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6f28-a7f6-4e03-89db-16007b0d5769 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.587581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1223/2435] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:56.588792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.589054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.589325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.591929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.592721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.593143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.593437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.594122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.595730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.609327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1213/2436] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:56.609327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.610665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:56.611866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.616600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:56.621222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1224/2437] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:56.621369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.625541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1865c80-71bc-4cf8-b4ff-811facf1db9a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.632010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41dc94-1404-403c-ae11-dcb03894da43 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.635092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.636136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.636404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.637206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.637915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.638440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:56.638638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.638745 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.639022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.639387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.640123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.640781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.641847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc906df-b03b-4176-b79e-788a39b46d45 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.652510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1225/2438] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:56.654076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.654466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.654832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.658156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1214/2439] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:56.659472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.671140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d1afa-0ee8-446d-b507-340f15044b18 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.676501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.677155 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.677426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.677729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] PATH_INFO: `/v3/projects/d450c75ccc0646c0a98d869609ecc2a9/users/2d80377cd1c84542bda5c0e8936e91e3/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.678314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:56.679592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.690031 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1226/2440] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:56.690973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.702692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb160b0-b96c-423a-91f9-b3e3cc1788f9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.705974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d450c75ccc0646c0a98d869609ecc2a9, user_id=2d80377cd1c84542bda5c0e8936e91e3, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.708792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.709503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.709650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.710087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.710456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.711180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.714704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c330045-6519-4c4c-8439-8f8ae88672bf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.743075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1215/2441] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:56 2022] PUT /identity/v3/projects/d450c75ccc0646c0a98d869609ecc2a9/users/2d80377cd1c84542bda5c0e8936e91e3/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:56.744117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.765125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1227/2442] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:56.769504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.769828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.770115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.801986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] MFA Rules not processed for user `2d80377cd1c84542bda5c0e8936e91e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:56.803305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.820398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50a6b0-deef-416f-9c49-ec663611056c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.843982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:56.873118 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1228/2443] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:56.874044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.883948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.884570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.884941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.885122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.885868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:29:56.886176 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.887841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.899701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1216/2444] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:29:56.900496 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.902930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:56.938354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ffc12-2bf8-4fe8-b238-bfe61f5a5f81 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.942904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.943508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.943783 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.944056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.944674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.946641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.960892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbcc-6dfa-4cf8-81f1-2406575844c7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.964721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.965326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.965621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.966504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.969309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1229/2445] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:56.970216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.975890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.977327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.979860 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a354d3-7cd9-4565-8b56-903889fecd8a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:56.985618 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:56.986218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:56.986500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:56.986779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:56.987016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1217/2446] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:56.987398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:56.987790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:56.989285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:56.999266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f3e3-fe07-4543-90a7-8f2d43d538ec service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.004256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.004856 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.005138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.005407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.006089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.007868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.011191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:57.022641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1230/2447] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:56 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:57.023703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1218/2448] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:57.027054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.030024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.040681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2cfc-c5aa-4e9f-b6ad-34ef2b649bd2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.110258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6e19-bfe1-4714-aa30-0e8ed2e2b030 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.114633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.115258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.115472 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.115598 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.115788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.116307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.116587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.116803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.116937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.117782 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.119691 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.120594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.139290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1219/2449] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:29:57.141668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.155564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1231/2450] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:29:57.156645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.162150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d696d8-25fb-44eb-a9a3-45c51f08a6e5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.167477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.168122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37272dcb-afe7-4903-a3f1-0df9b916e1db admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.168262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.168403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.169441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] PATH_INFO: `/v3/projects/b688aec09de646cd9b5242cffd4e2343/users/e6bcfcbeab6849729a32efb6580f6cef/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.174612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.175573 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.175862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.176143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.176809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.178493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.188333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b688aec09de646cd9b5242cffd4e2343, user_id=e6bcfcbeab6849729a32efb6580f6cef, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.191542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013d14b-cf16-4ed6-a71a-eb3bbb62dfa8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.192716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1232/2451] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:29:57.196878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.209511 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667870fb-bc97-4ca8-b603-d60967342650 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.212527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1220/2452] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:57 2022] PUT /identity/v3/projects/b688aec09de646cd9b5242cffd4e2343/users/e6bcfcbeab6849729a32efb6580f6cef/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:57.224250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.224652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.225025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.237679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:57.259451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] MFA Rules not processed for user `e6bcfcbeab6849729a32efb6580f6cef`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:57.260847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:57.303562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.304567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.304994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.305410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.306249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.308234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.320870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1233/2453] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:57.325278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.337117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8aaf-c88d-4d70-929d-f0a64cd9fa9d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.344313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.345099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.345384 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.345684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] PATH_INFO: `/v3/projects/d6c0e06fdff94b499e0f484f74bc6387/users/dd75249e72af40b7beb4e28db4ebf201/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.365837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1221/2454] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:57.367361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.369748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c0e06fdff94b499e0f484f74bc6387, user_id=dd75249e72af40b7beb4e28db4ebf201, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.374471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf23cff-a637-44f2-bf7b-ee1c633623c7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.381755 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3303a1-2669-41b2-8cf3-e1b27936fd35 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.388886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.389377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.389770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.390040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.390909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.392919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.393649 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1234/2455] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:57 2022] PUT /identity/v3/projects/d6c0e06fdff94b499e0f484f74bc6387/users/dd75249e72af40b7beb4e28db4ebf201/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:29:57.406433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.406829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.407278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.441678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] MFA Rules not processed for user `dd75249e72af40b7beb4e28db4ebf201`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:57.442815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:57.443990 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1222/2456] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:57.448944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.508422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:57.530847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1235/2457] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:57.531763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.537613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2274d5-b1c1-4c53-bdce-dc0ce198d97a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.543597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.544258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.544584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.544894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.545573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.545728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e33f68-18e4-4ba4-b856-1cdf40102924 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.547700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.551101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.551787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.552132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.552769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.553471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.555791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.576210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1223/2458] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:57.577414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.590334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98931a-d2f8-4a3e-ac04-1d7ac8430d93 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.597672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.598230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.598546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.598835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.599501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.601272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.618468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1224/2459] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:57.618468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.629886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76679b-3710-43d1-a58b-c795362a73a8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.636801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1236/2460] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:57.638061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.731189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f140-6041-4260-b060-5c35dfa51d73 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.736321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.736964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.737245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.737706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.738431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.740933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.751775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:57.774065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1237/2461] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:29:57 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:29:57.776208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.806744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69677-5486-4b69-a9b1-2f3eac1fea60 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.827358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.828245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.828777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.829157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.830121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.833080 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.864193 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1238/2462] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:57.865500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.891160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5bfb3-6ccf-483f-a2fb-569db486744c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:57.892656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.893633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.894815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.895200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] PATH_INFO: `/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d/users/b30e1b49aea14732b1c756154cec7531/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.897857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:57.898541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:57.898875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:57.899152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] PATH_INFO: `/v3/projects/41d58f9b1b6d45679985c525322f3223/users/689a1cf9078e44ba9803c757fbf97506/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:57.917688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d58f9b1b6d45679985c525322f3223, user_id=689a1cf9078e44ba9803c757fbf97506, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.918950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b8e50af14449aa9dfd630f1e0ef50d, user_id=b30e1b49aea14732b1c756154cec7531, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:57.920715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.921636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:57.940434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1239/2463] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:57 2022] PUT /identity/v3/projects/41d58f9b1b6d45679985c525322f3223/users/689a1cf9078e44ba9803c757fbf97506/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:57.941449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1225/2464] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:57 2022] PUT /identity/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d/users/b30e1b49aea14732b1c756154cec7531/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:57.945123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:57.945373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:58.024954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c190ad-600f-4e03-9b9f-c17c569e81c6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.031407 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.032029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.032458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.032712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.033403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.035471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.042107 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ced12-beb9-452b-a7f0-2236a7df7edf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.048687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1226/2465] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:29:58.049620 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.050357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.050700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.051041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.051962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.052876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:58.053790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.069205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1240/2466] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:29:58.069392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527c663-6d39-434b-9e37-aeb6a35d6ad1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.070510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:58.073110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.073812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.074091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.074365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] PATH_INFO: `/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d/users/b30e1b49aea14732b1c756154cec7531/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.079996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b8e50af14449aa9dfd630f1e0ef50d, user_id=b30e1b49aea14732b1c756154cec7531, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.082457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaa13c-d391-42fa-b720-3e8f7d8c1ab7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.083245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589311f3-e0e5-4fc1-9db5-86f6d91fe30c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.087699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.088417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.088693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.089362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] PATH_INFO: `/v3/projects/41d58f9b1b6d45679985c525322f3223/users/689a1cf9078e44ba9803c757fbf97506/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.091883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d58f9b1b6d45679985c525322f3223, user_id=689a1cf9078e44ba9803c757fbf97506, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.095371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd70fc0-660b-4667-a053-2eb7ccd55537 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.097489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1227/2467] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:58 2022] PUT /identity/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d/users/b30e1b49aea14732b1c756154cec7531/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:58.106926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1241/2468] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:29:58 2022] PUT /identity/v3/projects/41d58f9b1b6d45679985c525322f3223/users/689a1cf9078e44ba9803c757fbf97506/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:29:58.116158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.116158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.116158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.117994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.118529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.118900 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.152484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] MFA Rules not processed for user `689a1cf9078e44ba9803c757fbf97506`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:58.153819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:58.156074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] MFA Rules not processed for user `b30e1b49aea14732b1c756154cec7531`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:29:58.157266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:58.166859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:58.266166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1242/2469] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:29:58.283492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1228/2470] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:29:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:29:58.286286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:58.296699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:29:58.298138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:29:58.317841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:29:58.346861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b554a2-fc5e-4128-9752-6f1f28eb028a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.350574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.351182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.351464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.351733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.357033 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0438d0-8da8-4926-be86-8aa2fb2fff01 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:29:58.360911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:29:58.361514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:29:58.361837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:29:58.362117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:29:58.371085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.373094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.382833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1229/2471] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:29:58.414256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:29:58.415722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a7962-600a-4a92-8b6f-f9796b494cac service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:29:58.429249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1243/2472] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:29:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:29:59.941642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:00.026340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49b7a9-0d24-46b2-ae8b-72a5475e647e service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:00.031007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:00.031632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:00.031907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:00.032175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] PATH_INFO: `/v3/users/e6bcfcbeab6849729a32efb6580f6cef` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:00.033297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bcfcbeab6849729a32efb6580f6cef)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:00.035266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:00.084649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bcfcbeab6849729a32efb6580f6cef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.085311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bcfcbeab6849729a32efb6580f6cef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.085940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bcfcbeab6849729a32efb6580f6cef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.091347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:00.103081 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bcfcbeab6849729a32efb6580f6cef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.111988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bcfcbeab6849729a32efb6580f6cef'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.143138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1230/2473] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:29:59 2022] DELETE /identity/v3/users/e6bcfcbeab6849729a32efb6580f6cef => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:00.147522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:00.147963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:00.153422 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:00.183818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:00.185105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:00.260439 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1244/2474] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:00.271679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:00.276351 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:00.379581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf168331-9075-4f46-be7c-9c53fc6b7907 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:00.383567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:00.384250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:00.387966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:00.387966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:00.407290 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:00.408839 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:00.420724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1231/2475] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:00.712010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:00.713685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:00.733291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb3bc1-090d-462e-a5d0-95edcc4dd98b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:00.739178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:00.740695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:00.741016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:00.741233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] PATH_INFO: `/v3/projects/b688aec09de646cd9b5242cffd4e2343` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:00.742586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b688aec09de646cd9b5242cffd4e2343)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:00.744654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:00.793703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b688aec09de646cd9b5242cffd4e2343', 'request_id': 'req-73bddcff-dc30-441c-86c5-72a7bdb6e07d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.801405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b688aec09de646cd9b5242cffd4e2343', 'request_id': 'req-73bddcff-dc30-441c-86c5-72a7bdb6e07d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.807048 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b688aec09de646cd9b5242cffd4e2343', 'request_id': 'req-73bddcff-dc30-441c-86c5-72a7bdb6e07d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.818211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] The token cache is being invalidate because project b688aec09de646cd9b5242cffd4e2343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:00.818501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.819123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.820583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1245/2476] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:00 2022] DELETE /identity/v3/projects/b688aec09de646cd9b5242cffd4e2343 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:00.826226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:00.836637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b271eb-3fae-4110-9c2e-d6e050a2e94e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:00.894828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:00.895449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:00.895725 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:00.895994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] PATH_INFO: `/v3/users/b30e1b49aea14732b1c756154cec7531` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:00.897093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30e1b49aea14732b1c756154cec7531)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:00.899173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:00.945820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30e1b49aea14732b1c756154cec7531'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.946483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30e1b49aea14732b1c756154cec7531'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.947229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30e1b49aea14732b1c756154cec7531'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.959850 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30e1b49aea14732b1c756154cec7531'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.966685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45e5ed6c-107b-4764-a1af-48f5dcc11922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30e1b49aea14732b1c756154cec7531'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:00.968347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:00.982576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1232/2477] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:00 2022] DELETE /identity/v3/users/b30e1b49aea14732b1c756154cec7531 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:01.261563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.263138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:01.274416 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:01.279206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddcff-dc30-441c-86c5-72a7bdb6e07d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.351982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.352545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.352825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.353100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] PATH_INFO: `/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.355876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b8e50af14449aa9dfd630f1e0ef50d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.358550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.412967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b8e50af14449aa9dfd630f1e0ef50d', 'request_id': 'req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:01.422709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b8e50af14449aa9dfd630f1e0ef50d', 'request_id': 'req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:01.428878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b8e50af14449aa9dfd630f1e0ef50d', 'request_id': 'req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:01.441291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] The token cache is being invalidate because project 96b8e50af14449aa9dfd630f1e0ef50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:01.441583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:01.442306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a78589a7-6146-4cf4-a4a9-df920b9bcdc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:01.443998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1246/2478] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:01 2022] DELETE /identity/v3/projects/96b8e50af14449aa9dfd630f1e0ef50d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:01.448829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38328583-b05f-4725-87de-f271762a328f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.449108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38328583-b05f-4725-87de-f271762a328f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.449381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38328583-b05f-4725-87de-f271762a328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.468222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-38328583-b05f-4725-87de-f271762a328f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:01.469416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-38328583-b05f-4725-87de-f271762a328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:01.542019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1233/2479] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:01.550643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.550930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.551210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.572283 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:01.573525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:01.581464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1247/2480] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:01.585739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38328583-b05f-4725-87de-f271762a328f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.596234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38328583-b05f-4725-87de-f271762a328f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.600831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.601436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.601738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.602061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.602652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:01.602953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.604561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.616975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1234/2481] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:01.621329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.630504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d485-df4c-419f-804d-3d02814d9816 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.635150 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.635750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.636038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.636312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.636928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.638845 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.665953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1248/2482] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:01.670780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.680672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283591-7411-4fb4-bc4a-5412dff59723 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.736350 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.736978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.737241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.737512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.738167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.740078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.760535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1235/2483] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:01.764762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.775003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c632-1201-48ff-9446-1d13488523d6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.780149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.780748 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.781028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.781301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.781987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.783578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.793234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1249/2484] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:01.798124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.808201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1c16-7f5a-42c3-bf61-2f561b5209ad admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.815175 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.815796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.816660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.816660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.816984 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.818624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe2dee-85aa-4546-b6dd-4ccf818ae2cc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.830331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1236/2485] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:01.834579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:01.846291 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e72f-0cc8-482a-a1c7-34b637de26fc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:01.850111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:01.850771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.851049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.851327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] PATH_INFO: `/v3/projects/0f9968a19805449c863bb75c58bfc620/users/82a7bf60275a4aea8a3017b115e2a339/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.868219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9968a19805449c863bb75c58bfc620, user_id=82a7bf60275a4aea8a3017b115e2a339, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:01.870936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:01.883969 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1250/2486] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:01 2022] PUT /identity/v3/projects/0f9968a19805449c863bb75c58bfc620/users/82a7bf60275a4aea8a3017b115e2a339/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:01.893818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:01.894103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:01.894381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:01.922239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] MFA Rules not processed for user `82a7bf60275a4aea8a3017b115e2a339`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:01.923464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:01.968552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:01.991845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1237/2487] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:01.996294 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.030186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:02.066855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9b15-fa88-41b2-8f37-18703b7221c4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.071769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.072381 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.072670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.072945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.073563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.081894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.112171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1251/2488] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:01 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:02.115835 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.127410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f2613-cdf6-4408-9b34-fa22c9b2c34f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.197998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.198698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.199041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.199423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.200601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.202726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.228757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1238/2489] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:02 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:02.233803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.247706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f339e2-3e59-43b7-91a4-20b54965980e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.252951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.253868 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.254147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.254423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.255071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.256653 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.268323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1252/2490] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:02.272321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.283373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960895-927d-4152-8131-c762b449b861 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.288031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.288699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.289047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.289326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] PATH_INFO: `/v3/projects/65a9f63657cb4b1ab7d470a2695a398e/users/8bd6ca5a197242ddb47bf91a157e5bf8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.304593 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a9f63657cb4b1ab7d470a2695a398e, user_id=8bd6ca5a197242ddb47bf91a157e5bf8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.307298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.322812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1239/2491] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:02 2022] PUT /identity/v3/projects/65a9f63657cb4b1ab7d470a2695a398e/users/8bd6ca5a197242ddb47bf91a157e5bf8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:02.327621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.337741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c93af-5457-4906-b4d2-0589aeabb4fc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.395376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.395905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.396187 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.396486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.397124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.398741 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd90771-ef89-446d-9a9e-486708bcb6d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.408112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1253/2492] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:02.412359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.422821 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2c7a9-3310-42dc-a3d9-a4f1a44e1dfe admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.427413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.428062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.428338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.428606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] PATH_INFO: `/v3/projects/65a9f63657cb4b1ab7d470a2695a398e/users/8bd6ca5a197242ddb47bf91a157e5bf8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.430522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a9f63657cb4b1ab7d470a2695a398e, user_id=8bd6ca5a197242ddb47bf91a157e5bf8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.433156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.444419 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1240/2493] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:02 2022] PUT /identity/v3/projects/65a9f63657cb4b1ab7d470a2695a398e/users/8bd6ca5a197242ddb47bf91a157e5bf8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:02.465304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.465304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.465304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.490122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] MFA Rules not processed for user `8bd6ca5a197242ddb47bf91a157e5bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:02.491447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:02.565684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1254/2494] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:02.578720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.581120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:02.638619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6b508-877f-49f4-8fa2-12d38eb1e02e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.642437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:02.643045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:02.643321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:02.643595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:02.652833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:02.654227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:02.663440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1241/2495] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:02.985640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:02.994817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd19e8c-c035-45f5-9d6f-2a340a315054 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:02.999982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6ca5a197242ddb47bf91a157e5bf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65a9f63657cb4b1ab7d470a2695a398e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.000583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.000863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.001136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.001804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:03.002109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.003695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.014932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1255/2496] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:03.018797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:03.027734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8feb75-1ced-4667-a450-5c6cf169384a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:03.032213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6ca5a197242ddb47bf91a157e5bf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65a9f63657cb4b1ab7d470a2695a398e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.032791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.033063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.033336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.033972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.035912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.063161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1242/2497] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:03 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:03.067153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:03.076985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900bfb-555a-4170-9cb2-405bad1fcf14 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:03.085397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:03.135665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6ca5a197242ddb47bf91a157e5bf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65a9f63657cb4b1ab7d470a2695a398e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.136337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.136575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.136854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.137506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:03.137848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.139457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.152451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1256/2498] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:03.192545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:03.203024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e829d-787d-4e4d-a07f-043184ffd604 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:03.207748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6ca5a197242ddb47bf91a157e5bf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65a9f63657cb4b1ab7d470a2695a398e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.208373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.208671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.208925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] PATH_INFO: `/v3/projects/fd29fbb82c8e4f6f913249fe942dbbda` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.210075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fd29fbb82c8e4f6f913249fe942dbbda)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.211966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.236076 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:03.272345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd29fbb82c8e4f6f913249fe942dbbda', 'request_id': 'req-59934357-87da-4caa-a0b7-53f516149d81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.278657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd29fbb82c8e4f6f913249fe942dbbda', 'request_id': 'req-59934357-87da-4caa-a0b7-53f516149d81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.286620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd29fbb82c8e4f6f913249fe942dbbda', 'request_id': 'req-59934357-87da-4caa-a0b7-53f516149d81'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.298580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] The token cache is being invalidate because project fd29fbb82c8e4f6f913249fe942dbbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:03.298866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.299507 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-59934357-87da-4caa-a0b7-53f516149d81 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.300992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1243/2499] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:03 2022] DELETE /identity/v3/projects/fd29fbb82c8e4f6f913249fe942dbbda => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:03.492925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:03.496352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:03.568558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5cc02-ee89-428b-a174-79b885e21c78 tempest-VolumeQuotaClassesTest-1843028250 tempest-VolumeQuotaClassesTest-1843028250-project-admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:03.622935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.623626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.623901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.624172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] PATH_INFO: `/v3/users/82a7bf60275a4aea8a3017b115e2a339` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.625270 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a7bf60275a4aea8a3017b115e2a339)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.627205 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.675771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a7bf60275a4aea8a3017b115e2a339'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.682936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a7bf60275a4aea8a3017b115e2a339'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.683749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:03.702268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a7bf60275a4aea8a3017b115e2a339'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.711035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a7bf60275a4aea8a3017b115e2a339'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.711662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a7bf60275a4aea8a3017b115e2a339'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:03.719992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1257/2500] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:03 2022] DELETE /identity/v3/users/82a7bf60275a4aea8a3017b115e2a339 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:03.724413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.724702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.724978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.743567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:03.744799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:03.805771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1244/2501] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:03.816342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:03.817906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:03.875064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ceb26-6a98-43a4-ad03-b1b86081ae68 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:03.878752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:03.879354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:03.879635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:03.879908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:03.889425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:03.890873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:03.900923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1258/2502] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:30:04.162514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:04.163994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:04.176607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ad21f-dc55-4bf9-99e5-458112ac03d2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:04.181255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:04.181941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:04.182221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:04.182566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] PATH_INFO: `/v3/projects/0f9968a19805449c863bb75c58bfc620` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:04.183738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9968a19805449c863bb75c58bfc620)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:04.186587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:04.233359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9968a19805449c863bb75c58bfc620', 'request_id': 'req-ffd60e33-402c-44f5-a367-57693b8836bc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.238872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9968a19805449c863bb75c58bfc620', 'request_id': 'req-ffd60e33-402c-44f5-a367-57693b8836bc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.246911 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9968a19805449c863bb75c58bfc620', 'request_id': 'req-ffd60e33-402c-44f5-a367-57693b8836bc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.258521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] The token cache is being invalidate because project 0f9968a19805449c863bb75c58bfc620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:04.258805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.259433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.260890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1245/2503] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:04 2022] DELETE /identity/v3/projects/0f9968a19805449c863bb75c58bfc620 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:04.264944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:04.276847 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0b589-3a7e-48ad-9cb7-6e635dd613ba service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:04.351730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:04.352368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:04.352652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:04.352930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] PATH_INFO: `/v3/users/8bd6ca5a197242ddb47bf91a157e5bf8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:04.354087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd6ca5a197242ddb47bf91a157e5bf8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:04.356028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:04.401041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd6ca5a197242ddb47bf91a157e5bf8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.407875 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd6ca5a197242ddb47bf91a157e5bf8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.420006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd6ca5a197242ddb47bf91a157e5bf8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.423578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:04.427435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd6ca5a197242ddb47bf91a157e5bf8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.428491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed3b4b15-1e5b-44f2-963c-6e01ac475298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd6ca5a197242ddb47bf91a157e5bf8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.436637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1259/2504] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:04 2022] DELETE /identity/v3/users/8bd6ca5a197242ddb47bf91a157e5bf8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:04.706983 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:04.708449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:04.722098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd60e33-402c-44f5-a367-57693b8836bc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:04.747829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:04.781128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:04.781760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:04.782090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:04.782320 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] PATH_INFO: `/v3/projects/65a9f63657cb4b1ab7d470a2695a398e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:04.783454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a9f63657cb4b1ab7d470a2695a398e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:04.785366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:04.832869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a9f63657cb4b1ab7d470a2695a398e', 'request_id': 'req-b662a498-779e-4754-8146-f4d05bd3e17c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.838449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a9f63657cb4b1ab7d470a2695a398e', 'request_id': 'req-b662a498-779e-4754-8146-f4d05bd3e17c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.846647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a9f63657cb4b1ab7d470a2695a398e', 'request_id': 'req-b662a498-779e-4754-8146-f4d05bd3e17c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.858489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] The token cache is being invalidate because project 65a9f63657cb4b1ab7d470a2695a398e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:04.858781 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.859432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b662a498-779e-4754-8146-f4d05bd3e17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:04.860922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1246/2505] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:04 2022] DELETE /identity/v3/projects/65a9f63657cb4b1ab7d470a2695a398e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:04.865824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:04.866307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:04.866415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:04.887089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:04.888355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:04.949685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1260/2506] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:04.957969 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:04.958257 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:04.958531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:04.978607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:04.979811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:04.987795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1247/2507] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:04.991962 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.002520 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22f4dd-97a1-4e2c-a643-9b015238ffe8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.007082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.007677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.007954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.008223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.008861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:05.009162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.010796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.024418 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1261/2508] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:05.028788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.041766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b49acf-f48d-4b81-97a3-53e8787b9760 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.047034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.047696 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.047996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.048230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.048869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.050795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.078334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1248/2509] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:05 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:05.083440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.094811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a5f1d-59fd-446a-ba59-9ea88641f7ea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.164430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.165051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.165337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.165641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.166278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.168198 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.190344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1262/2510] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:05 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:05.194474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.205129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52e247-5316-4aaf-b21c-83edb39942c0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.209890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.210497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.210773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.211050 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.211694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.213289 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.224543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1249/2511] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:05.228685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.240026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270217-81d1-43e4-ac7f-d982aa20a961 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.245320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.246071 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.246333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.246629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.247334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.249099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.261183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1263/2512] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:05.265462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.265688 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.272377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:05.281861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af583ca-da35-4a15-8c55-d1dd3e265fa3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.288579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.289353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.289629 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.289932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] PATH_INFO: `/v3/projects/1979fec18283457995dde2e3f6beb4fc/users/faf14ef6dd6646c6872d057526605320/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.308428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1979fec18283457995dde2e3f6beb4fc, user_id=faf14ef6dd6646c6872d057526605320, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.312216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc896e0-c96f-4b12-980f-ab981c1ccf91 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.339024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1264/2513] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:05 2022] PUT /identity/v3/projects/1979fec18283457995dde2e3f6beb4fc/users/faf14ef6dd6646c6872d057526605320/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:05.342159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.342493 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.342807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.369220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09b55-afaa-46e2-82e0-1e38dea00bca admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.384299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] MFA Rules not processed for user `faf14ef6dd6646c6872d057526605320`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:05.385640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:05.388041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:05.421890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:05.481484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1265/2514] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:05.486822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.488388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.489295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.489898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.498454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.498454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:05.511619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.513271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.525657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1250/2515] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:05.569159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57fa32-6b30-4473-b3d8-a33f9ea82d90 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.575088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.576799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.577082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.577354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.578449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.580392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.625075 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1266/2516] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:05 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:05.635402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.650810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42761-7c54-4d32-b948-b586f5981ee3 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.741811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.742615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.742993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.750416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.750879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.755422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.818999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1251/2517] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:05 2022] POST /identity/v3/users => generated 303 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:05.820595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.836562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4e-6061-4272-9873-5a49fdc7d005 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.844331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.845172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.845304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.845681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.846880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.848797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.865725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1267/2518] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:05.868080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.881962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7d3f6-a54d-41ac-bea3-690a3e8cd3ad admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:05.888342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:05.888974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:05.889252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:05.889519 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] PATH_INFO: `/v3/projects/2b73d514c30640009feda21d101a5fbc/users/7ac67f16c44f4567aa9b6c975f45dfb8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:05.918447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b73d514c30640009feda21d101a5fbc, user_id=7ac67f16c44f4567aa9b6c975f45dfb8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:05.922003 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:05.939708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1252/2519] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:05 2022] PUT /identity/v3/projects/2b73d514c30640009feda21d101a5fbc/users/7ac67f16c44f4567aa9b6c975f45dfb8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:05.944562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:05.959350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0da6e-1b69-4d7b-b34e-9cff2658f347 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.059499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.061300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.061764 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.062426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.063414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.065410 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a037b7-81fe-4611-b2d6-3f9a028e4a72 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.079855 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1268/2520] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:06.084285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:06.101146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d338f-28b1-4157-bbff-ac2487e62ff4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.106555 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.108013 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.108297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.108572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] PATH_INFO: `/v3/projects/2b73d514c30640009feda21d101a5fbc/users/7ac67f16c44f4567aa9b6c975f45dfb8/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.110724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b73d514c30640009feda21d101a5fbc, user_id=7ac67f16c44f4567aa9b6c975f45dfb8, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.113423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.128837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1253/2521] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:06 2022] PUT /identity/v3/projects/2b73d514c30640009feda21d101a5fbc/users/7ac67f16c44f4567aa9b6c975f45dfb8/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:06.140373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.140669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.140953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.173666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] MFA Rules not processed for user `7ac67f16c44f4567aa9b6c975f45dfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:06.175026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.210370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:06.255664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1269/2522] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:06.266718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:06.268447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.336475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da1c7-a696-4f1f-a9ee-058d76b4b104 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.340629 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.341259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.342299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.342601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.350193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:06.353078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.355187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.367078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1254/2523] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:06.418886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:06.420975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.433128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f72d-74e6-4c04-b995-10417a727997 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.437450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.438102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.438386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.438661 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.498908 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.500849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d258cc9-3e56-4ea3-b2a6-bd342ab3cdf9 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.515828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1270/2524] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:06.536914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:06.538433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.608526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d0fc-a1ce-4bae-89ef-bf6a6ec7a49a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.613131 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.613787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.614062 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.614329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] PATH_INFO: `/v3/users/2d80377cd1c84542bda5c0e8936e91e3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.615457 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d80377cd1c84542bda5c0e8936e91e3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.617364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.667297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d80377cd1c84542bda5c0e8936e91e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:06.667957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d80377cd1c84542bda5c0e8936e91e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:06.668573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d80377cd1c84542bda5c0e8936e91e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:06.684464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d80377cd1c84542bda5c0e8936e91e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:06.693410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d80377cd1c84542bda5c0e8936e91e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:06.708071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1255/2525] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:06 2022] DELETE /identity/v3/users/2d80377cd1c84542bda5c0e8936e91e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:06.712642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.712939 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.713219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.733551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:06.734856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.804056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1271/2526] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:06.815429 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:06.817063 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:06.893825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf7a57-32cb-4bc5-8ba1-e75872098f28 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:06.898546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:06.899229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:06.899522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:06.899999 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:06.910761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:06.912428 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:06.924522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1256/2527] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:07.228406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.232487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.257450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218bff2-61ed-4040-8856-6daa397af8c1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.264474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.265290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.265886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.266446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] PATH_INFO: `/v3/projects/d450c75ccc0646c0a98d869609ecc2a9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.277658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d450c75ccc0646c0a98d869609ecc2a9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.277658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.295400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.296917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.327940 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:07.345341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd450c75ccc0646c0a98d869609ecc2a9', 'request_id': 'req-d4ac9564-5dbd-4eed-84a1-e42b64a54040'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:07.356685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd450c75ccc0646c0a98d869609ecc2a9', 'request_id': 'req-d4ac9564-5dbd-4eed-84a1-e42b64a54040'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:07.360859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:07.368484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd450c75ccc0646c0a98d869609ecc2a9', 'request_id': 'req-d4ac9564-5dbd-4eed-84a1-e42b64a54040'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:07.391017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] The token cache is being invalidate because project d450c75ccc0646c0a98d869609ecc2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:07.391320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:07.393105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d4ac9564-5dbd-4eed-84a1-e42b64a54040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:07.395581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73da86-1582-48f9-96fc-c4090d87e3be service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.396154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1272/2528] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:07 2022] DELETE /identity/v3/projects/d450c75ccc0646c0a98d869609ecc2a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:07.402074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3c95-f160-4757-b842-e44f38cefe92 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.402774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3c95-f160-4757-b842-e44f38cefe92 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.403129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f3c95-f160-4757-b842-e44f38cefe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.431276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-053f3c95-f160-4757-b842-e44f38cefe92 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:07.432526 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-053f3c95-f160-4757-b842-e44f38cefe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.469627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.470246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.470594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.470786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.472701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.497103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1273/2529] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:07.506846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.507139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.507423 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.526946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:07.528169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.536460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1274/2530] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:07.536586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.538072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.541728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.550278 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1257/2531] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:30:07.551929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63bc42-36cd-47b9-8309-ad796bcbed0d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.556574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.557178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.557459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.557792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.558492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:07.558727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.561235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.573046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1275/2532] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:30:07.577609 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.588514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f547cf-d9ea-4b5a-a19e-e4c2897d1da3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.594650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.595341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.595689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.596057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.596791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.598791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.627982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1258/2533] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:07 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:07.632066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.642792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ff99-ff28-4d32-acca-2232a783042c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.706341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.706956 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.707237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.707513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.708296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.710241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.734386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1276/2534] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:07 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:07.738591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.749633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39448946-2a57-4097-b02b-8c3f0614399c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.756541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.757162 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.757439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.757750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.758398 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.760042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.776925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1259/2535] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:07.781648 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.793688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb28c2-0026-4798-b84d-ee23555c1db7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.800181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.800799 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.801083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.801361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.802051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.803651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2c8f9-4461-4b59-80f1-2e12b5d6f60e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.816200 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1277/2536] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:07.820708 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.831972 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c5f9-bc87-497c-ba4a-c24595f797c2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:07.838655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:07.838655 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.839179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.839179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] PATH_INFO: `/v3/projects/6c01cdc6c55b4fdf999fbc75c1f4b51b/users/3e7526ef9eb0426a855f5b2b9be06a8d/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.856414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c01cdc6c55b4fdf999fbc75c1f4b51b, user_id=3e7526ef9eb0426a855f5b2b9be06a8d, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:07.860046 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:07.879055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1260/2537] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:07 2022] PUT /identity/v3/projects/6c01cdc6c55b4fdf999fbc75c1f4b51b/users/3e7526ef9eb0426a855f5b2b9be06a8d/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:07.884223 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:07.884447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:07.885086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:07.911842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:07.913237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:07.964488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:07.987184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1278/2538] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:07.997917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:07.999817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:08.034021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:08.062604 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4486ad-5bf8-4314-95f6-81000f5fc24d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:08.066512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:08.067121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:08.067395 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:08.067669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:08.078668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:08.080110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1115b2a-df44-4949-8973-02dd474ac0bb service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:08.090682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1261/2539] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:08.433222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:08.434856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:08.448574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997f887-254b-4eb8-a131-b9305b540da2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:08.454597 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:08.455309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:08.455658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:08.456010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] PATH_INFO: `/v3/users/d4f808f6f19243e3823ec53c12bc0af9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:08.457376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f808f6f19243e3823ec53c12bc0af9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:08.459485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:08.528535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f808f6f19243e3823ec53c12bc0af9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:08.537500 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f808f6f19243e3823ec53c12bc0af9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:08.554630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f808f6f19243e3823ec53c12bc0af9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:08.564839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f808f6f19243e3823ec53c12bc0af9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:08.565546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f808f6f19243e3823ec53c12bc0af9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:08.573798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1279/2540] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:08 2022] DELETE /identity/v3/users/d4f808f6f19243e3823ec53c12bc0af9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:08.577927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:08.578727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:08.579002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:08.598825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:08.600048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:08.700812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1262/2541] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:08.710370 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:08.719731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:08.821876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902bdd8-59e7-4da0-ba84-b8bb5977ee02 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:08.827115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:08.827755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:08.828117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:08.829009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:08.854731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:08.856808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:08.877271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1280/2542] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:09.824590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:09.826760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:09.839036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c37e-d5c6-428b-8e57-aa6676ee4231 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:09.844101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:09.844749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:09.845026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:09.845296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] PATH_INFO: `/v3/projects/280177513f744247a5c5aa1e2021dc68` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:09.846849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=280177513f744247a5c5aa1e2021dc68)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:09.848914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:09.945665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '280177513f744247a5c5aa1e2021dc68', 'request_id': 'req-351de17d-3fa5-4e00-8816-111317b02859'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:09.965150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '280177513f744247a5c5aa1e2021dc68', 'request_id': 'req-351de17d-3fa5-4e00-8816-111317b02859'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:09.991882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '280177513f744247a5c5aa1e2021dc68', 'request_id': 'req-351de17d-3fa5-4e00-8816-111317b02859'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.013184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] The token cache is being invalidate because project 280177513f744247a5c5aa1e2021dc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:10.014253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.015252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.017205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1263/2543] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:09 2022] DELETE /identity/v3/projects/280177513f744247a5c5aa1e2021dc68 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:10.020972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:10.042220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18190475-377c-4d0d-989d-0235d5e504e4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:10.207865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:10.208914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:10.209207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:10.209532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] PATH_INFO: `/v3/users/442e2ecb7bc84a4bb998cde9e95931cd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:10.211531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442e2ecb7bc84a4bb998cde9e95931cd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:10.215997 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:10.219081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:10.298469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442e2ecb7bc84a4bb998cde9e95931cd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.309692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:10.312792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:10.316669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442e2ecb7bc84a4bb998cde9e95931cd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.330410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351de17d-3fa5-4e00-8816-111317b02859 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:10.333010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442e2ecb7bc84a4bb998cde9e95931cd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.342986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:10.343632 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:10.343968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:10.344250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] PATH_INFO: `/v3/users/faf14ef6dd6646c6872d057526605320` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:10.346222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf14ef6dd6646c6872d057526605320)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:10.350955 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442e2ecb7bc84a4bb998cde9e95931cd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.351643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-db19068f-59ed-4eee-a3c2-9dbb8c7281f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442e2ecb7bc84a4bb998cde9e95931cd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.359647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:10.365079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1281/2544] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:10 2022] DELETE /identity/v3/users/442e2ecb7bc84a4bb998cde9e95931cd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:10.431028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf14ef6dd6646c6872d057526605320'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.431862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf14ef6dd6646c6872d057526605320'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.432970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf14ef6dd6646c6872d057526605320'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.436077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:10.489757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf14ef6dd6646c6872d057526605320'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.511585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf14ef6dd6646c6872d057526605320'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:10.563912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1264/2545] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:10 2022] DELETE /identity/v3/users/faf14ef6dd6646c6872d057526605320 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:10.565901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:10.566293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:10.567353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:10.597482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:10.599538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:10.706246 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1282/2546] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:10.730128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:10.731887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:10.824578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf82f8-8d78-4dcc-b958-0391f4339e45 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:10.829584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:10.830332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:10.830691 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:10.831035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:10.845077 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:10.863441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:10.886211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1265/2547] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:11.504679 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:11.506400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:11.540212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c79b2-915e-42da-92b7-06340ab2a90b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:11.568066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:11.568728 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:11.569028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:11.569290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] PATH_INFO: `/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:11.584682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4388ae1d9f40bc9d0754e1b8a62f6f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:11.586883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:11.698689 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:11.700430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:11.757838 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fad95-6843-429f-ae35-68a4b52b06d3 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:11.808711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4388ae1d9f40bc9d0754e1b8a62f6f', 'request_id': 'req-86dd843c-ada7-4028-bd7d-d3e260704e3e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:11.846637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4388ae1d9f40bc9d0754e1b8a62f6f', 'request_id': 'req-86dd843c-ada7-4028-bd7d-d3e260704e3e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:11.876702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4388ae1d9f40bc9d0754e1b8a62f6f', 'request_id': 'req-86dd843c-ada7-4028-bd7d-d3e260704e3e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:11.929268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:11.932403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] The token cache is being invalidate because project 1a4388ae1d9f40bc9d0754e1b8a62f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:11.932716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:11.933733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-86dd843c-ada7-4028-bd7d-d3e260704e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:11.937016 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:11.940451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:11.940771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] PATH_INFO: `/v3/projects/1979fec18283457995dde2e3f6beb4fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:11.946578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1283/2548] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:11 2022] DELETE /identity/v3/projects/1a4388ae1d9f40bc9d0754e1b8a62f6f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:11.952495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1979fec18283457995dde2e3f6beb4fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:11.953496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538adaf-f1cf-4767-b16d-079a54781839 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:11.954511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:11.954923 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538adaf-f1cf-4767-b16d-079a54781839 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:11.959380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538adaf-f1cf-4767-b16d-079a54781839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:11.996453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:12.036766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6538adaf-f1cf-4767-b16d-079a54781839 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:12.040075 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1979fec18283457995dde2e3f6beb4fc', 'request_id': 'req-46810bb2-7652-4312-bad3-8df2d7f9198a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.049171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6538adaf-f1cf-4767-b16d-079a54781839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:12.050538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1979fec18283457995dde2e3f6beb4fc', 'request_id': 'req-46810bb2-7652-4312-bad3-8df2d7f9198a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.064692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1979fec18283457995dde2e3f6beb4fc', 'request_id': 'req-46810bb2-7652-4312-bad3-8df2d7f9198a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.084079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:12.111736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] The token cache is being invalidate because project 1979fec18283457995dde2e3f6beb4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:12.115987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.117318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.121216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1266/2549] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:11 2022] DELETE /identity/v3/projects/1979fec18283457995dde2e3f6beb4fc => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:12.125430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.127607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:12.221393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1284/2550] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:12.223347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.223637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.223919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.270918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:12.272207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:12.289719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1285/2551] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:12.290698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.309027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6240be6-edc5-482e-9299-e1a238a965e6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.324852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.325552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.325904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.326371 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.326838 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:12.327145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.328771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.352619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46810bb2-7652-4312-bad3-8df2d7f9198a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.357945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1286/2552] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:30:12.359168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.364019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.364886 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.365325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.365790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] PATH_INFO: `/v3/users/7ac67f16c44f4567aa9b6c975f45dfb8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.367128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac67f16c44f4567aa9b6c975f45dfb8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.374803 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4a55-3393-4d45-a5ac-dd32a7495ca7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.376686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.392191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.392459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.392747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.393022 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.393681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.401216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.467347 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1287/2553] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:12 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:12.471250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.510159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac67f16c44f4567aa9b6c975f45dfb8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.512469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac67f16c44f4567aa9b6c975f45dfb8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.515112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac67f16c44f4567aa9b6c975f45dfb8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.539638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac67f16c44f4567aa9b6c975f45dfb8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.553586 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac67f16c44f4567aa9b6c975f45dfb8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:12.582138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1267/2554] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:12 2022] DELETE /identity/v3/users/7ac67f16c44f4567aa9b6c975f45dfb8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:12.623348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d833-58ec-4dc8-9a5e-d0182417caa4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.756312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.756932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.757215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.757481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.758165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.761824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.798787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:12.812779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1288/2555] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:12 2022] POST /identity/v3/users => generated 293 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:12.814760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.818874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:12.846610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70307f23-449b-4130-8bb6-c96c3126eb49 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.851600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.852581 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.852861 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.853132 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.853919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.856122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.869307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1268/2556] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:12.877476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.905619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870e2d5-4688-4e71-903b-2810e4b55c8a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.916247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.923141 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:12.924027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.924998 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.925482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.926151 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.927242 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.931325 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.949642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb0ac-f3cf-488c-82a9-f4dceb93c9a3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.959614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1289/2557] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:12.963538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.963793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:12.964080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:12.964366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] PATH_INFO: `/v3/users/dd75249e72af40b7beb4e28db4ebf201` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:12.964965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:12.966093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd75249e72af40b7beb4e28db4ebf201)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:12.968408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:12.986271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f62ec-d329-42b4-9f60-feb9736cb79d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:12.998859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:12.999684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.000331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.000951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] PATH_INFO: `/v3/projects/17f0779cc2d34ff9a012b0d04361d514/users/4c4b11da98284bb6a4cbc41e993cb10a/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.036373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f0779cc2d34ff9a012b0d04361d514, user_id=4c4b11da98284bb6a4cbc41e993cb10a, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.039177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86defc-8cd0-4631-927d-02f07e624c9f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.041754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd75249e72af40b7beb4e28db4ebf201'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.042782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd75249e72af40b7beb4e28db4ebf201'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.043615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd75249e72af40b7beb4e28db4ebf201'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.059816 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:13.067243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1290/2558] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:12 2022] PUT /identity/v3/projects/17f0779cc2d34ff9a012b0d04361d514/users/4c4b11da98284bb6a4cbc41e993cb10a/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:13.068579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd75249e72af40b7beb4e28db4ebf201'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.083992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ebe5d4c8-59df-49d0-aad7-b2760823642d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd75249e72af40b7beb4e28db4ebf201'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.087160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.087449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.087722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.110824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1269/2559] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:12 2022] DELETE /identity/v3/users/dd75249e72af40b7beb4e28db4ebf201 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:13.120070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.120495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.120775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.126329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] MFA Rules not processed for user `4c4b11da98284bb6a4cbc41e993cb10a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:13.127922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.173337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:13.175645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.238705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1291/2560] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:13.239601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.240945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.306903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1270/2561] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:13.307980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.312225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.339532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1cbf-133e-401b-9886-b972a5e5c7e4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.348261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.349037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.349414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.349899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.350588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.352599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.390652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64346b1-447c-4d13-b74d-6a702d82f696 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.396834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.397727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.398001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.398275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] PATH_INFO: `/v3/projects/2b73d514c30640009feda21d101a5fbc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.399490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b73d514c30640009feda21d101a5fbc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.400017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1271/2562] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:13.401086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.401554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.403392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.488035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b73d514c30640009feda21d101a5fbc', 'request_id': 'req-d87b95e6-dd41-4372-9220-f9c1320fa302'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.494218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:13.501518 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b73d514c30640009feda21d101a5fbc', 'request_id': 'req-d87b95e6-dd41-4372-9220-f9c1320fa302'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.514343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b73d514c30640009feda21d101a5fbc', 'request_id': 'req-d87b95e6-dd41-4372-9220-f9c1320fa302'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.516474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abb9ce-f50f-4b6d-984a-3f81f3ff787c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.531449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] The token cache is being invalidate because project 2b73d514c30640009feda21d101a5fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:13.531887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.533053 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:13.534948 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1292/2563] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:13 2022] DELETE /identity/v3/projects/2b73d514c30640009feda21d101a5fbc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:13.535751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.542571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.590107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.590756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.591037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.592172 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.595008 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.634597 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:13.635400 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b95e6-dd41-4372-9220-f9c1320fa302 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.643752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.645682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.646963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.647492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.648617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.650756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161aee-b1a4-4838-abcd-5b14aa033a20 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.688587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.691729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.709036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1293/2564] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:13.710338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b1ba-df1f-4206-8653-b7f6561c524e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.710619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b1ba-df1f-4206-8653-b7f6561c524e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.710888 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b1ba-df1f-4206-8653-b7f6561c524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.711495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1272/2565] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:13.713944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.736287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db94d7-4c4c-4720-b548-74b42d6320a0 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.747771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.748414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.748692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.748966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.751259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.752936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.787529 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-55b3b1ba-df1f-4206-8653-b7f6561c524e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:13.789876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1273/2566] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:13.793720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.795253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-55b3b1ba-df1f-4206-8653-b7f6561c524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.824357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3799-67c5-4d9b-8b28-ab370dbb4867 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.824644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1294/2567] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:13.832595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.832773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.833024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.851534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.852928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.853625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.854079 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.855319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.858561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:13.861539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:13.863681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.879115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1295/2568] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:13.883919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.892352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1274/2569] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:13.895786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:13.921366 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860628c6-8dae-4c2d-9751-914be26048b3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.931607 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc212f-6f55-4ab9-8809-5e5a0fb31a3d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:13.937409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.939519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.940222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.941122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.942571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:13.942882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:13.945744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:13.949070 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:13.949895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:13.950578 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:13.952789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] PATH_INFO: `/v3/projects/52af5ef28ca347c0a68b3a2736278409/users/60df787466da4d95a3d9d85c6ccc128b/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:13.994539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1296/2570] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:13.999954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.006840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52af5ef28ca347c0a68b3a2736278409, user_id=60df787466da4d95a3d9d85c6ccc128b, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.009576 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944149c-de86-41ef-9921-d28d840f6d0a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.030181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113978-a117-4055-9ec3-6482b7d32cd3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.045286 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1275/2571] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:13 2022] PUT /identity/v3/projects/52af5ef28ca347c0a68b3a2736278409/users/60df787466da4d95a3d9d85c6ccc128b/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:14.048514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.049230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.050071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.099653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] MFA Rules not processed for user `60df787466da4d95a3d9d85c6ccc128b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:14.104117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.181896 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.182543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.182831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.183115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.183762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.190796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.224521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1276/2572] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:14.235364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.267600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f455c0e-2125-4f57-a4f4-ffbe627321ad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.277547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1297/2573] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:13 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:14.278643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.280450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.281384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.282448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.282907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.283757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.286174 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.287946 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.355921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1277/2574] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:14.357436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.389525 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:14.426819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:14.460556 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d151864-9775-4124-bb7f-4546ae781fea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.462578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ae7e-ba2f-4573-ab45-0fd6eada246b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.465805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.466455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.466738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.467011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.467646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.469589 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.475348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.476017 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.476299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.476569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] PATH_INFO: `/v3/projects/d6c0e06fdff94b499e0f484f74bc6387` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.478279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c0e06fdff94b499e0f484f74bc6387)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.480341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.502211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1278/2575] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:14.503056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.520746 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a64d-908b-4793-b75c-bf0c5e96b7e4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.530636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.531430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.531712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.531979 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.532828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.536297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d5d3-29a4-4e63-bdeb-cbe651f5bd60 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.561704 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c0e06fdff94b499e0f484f74bc6387', 'request_id': 'req-e578d732-c3f0-4705-93c7-d97d669e0420'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.575207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c0e06fdff94b499e0f484f74bc6387', 'request_id': 'req-e578d732-c3f0-4705-93c7-d97d669e0420'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.579327 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1279/2576] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/users => generated 300 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:14.584924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.585508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.585977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.593255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c0e06fdff94b499e0f484f74bc6387', 'request_id': 'req-e578d732-c3f0-4705-93c7-d97d669e0420'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.607245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] The token cache is being invalidate because project d6c0e06fdff94b499e0f484f74bc6387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:14.607444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.608334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.614047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1298/2577] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:14 2022] DELETE /identity/v3/projects/d6c0e06fdff94b499e0f484f74bc6387 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:14.614773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.620730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.629140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] MFA Rules not processed for user `3e7526ef9eb0426a855f5b2b9be06a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:14.631640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.710050 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d732-c3f0-4705-93c7-d97d669e0420 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.722544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.723263 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.723920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.724272 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.725706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.727553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.734711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1280/2578] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:14.735783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.737123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.742074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1299/2579] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:14.743081 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.745209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.752748 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e16115-ab10-4ce6-9ea4-3da49dd611a1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.757460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.758245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.759476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.759476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.760608 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.762106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.762225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ecc8-d190-4c85-8f05-55bbb75c9d03 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.766973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.767959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.768340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.768723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] PATH_INFO: `/v3/users/689a1cf9078e44ba9803c757fbf97506` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.769999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=689a1cf9078e44ba9803c757fbf97506)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.772428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.772558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1281/2580] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:14.773066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.774827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.789458 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab0e1-f9f2-47c5-a9d1-3bf9fc37e970 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.794927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.795523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.795806 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.796152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:14.797146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:14.799108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:14.833172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '689a1cf9078e44ba9803c757fbf97506'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.835509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1282/2581] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:14 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:14.836336 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.840854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:14.844864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '689a1cf9078e44ba9803c757fbf97506'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.860781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '689a1cf9078e44ba9803c757fbf97506'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.872337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '689a1cf9078e44ba9803c757fbf97506'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.873665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '689a1cf9078e44ba9803c757fbf97506'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:14.880852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1300/2582] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:14 2022] DELETE /identity/v3/users/689a1cf9078e44ba9803c757fbf97506 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:14.881828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:14.883209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:14.914896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa096a5-3803-45ef-b947-c213b819c439 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.941362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:14.969202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9f978-52d4-4aad-82cc-04acc6f4d618 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:14.979936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:14.981177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:14.981557 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:14.982535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] PATH_INFO: `/v3/projects/3727f950e6b0419292b928912891526a/users/03a483f7a1474b25b2ecfb007e82c33d/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.002456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.003584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.003584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.003785 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.004251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.005887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.007248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3727f950e6b0419292b928912891526a, user_id=03a483f7a1474b25b2ecfb007e82c33d, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.010815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.032738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1283/2583] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:15.032738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.035055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.067143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84d4b6-e86f-437c-867b-2995756ff30f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.071203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1301/2584] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:14 2022] PUT /identity/v3/projects/3727f950e6b0419292b928912891526a/users/03a483f7a1474b25b2ecfb007e82c33d/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:15.072298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.075950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.154932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.155567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.155843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.156116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.156954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.159160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.186361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589cd16-cd99-4fb8-9bcb-584d543e6965 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.194863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.195813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.196237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.196671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] PATH_INFO: `/v3/projects/bfeeb93d1df94eb38ee5fb54d5c459de/users/9b43bd85a0fb429c9af5ef8de7ee818e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.202825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1284/2585] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:15 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:15.203892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.214106 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfeeb93d1df94eb38ee5fb54d5c459de, user_id=9b43bd85a0fb429c9af5ef8de7ee818e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.218213 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.219802 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8272af3-be75-4bb0-b804-c8ca88cdf28a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.227207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.228036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.228415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.228871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.229808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.231562 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c0e4-ec5b-43c5-bb02-49efc9bd5a0d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.240063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1302/2586] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:15 2022] PUT /identity/v3/projects/bfeeb93d1df94eb38ee5fb54d5c459de/users/9b43bd85a0fb429c9af5ef8de7ee818e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:15.240972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.251086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1285/2587] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:15.252375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.252663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.252937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.287005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] MFA Rules not processed for user `9b43bd85a0fb429c9af5ef8de7ee818e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:15.288744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.318631 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:15.351054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222647-6484-4e6b-a51a-526998df00b6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.358494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.359219 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.359504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.359781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.360442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.362102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.373537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1303/2588] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:15.374628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.382779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1286/2589] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:15.383644 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.386305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935104a-7a2d-4917-9080-baa8f88752bf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.387717 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.392145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.392894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.393244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.393638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] PATH_INFO: `/v3/projects/3727f950e6b0419292b928912891526a/users/03a483f7a1474b25b2ecfb007e82c33d/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.395762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3727f950e6b0419292b928912891526a, user_id=03a483f7a1474b25b2ecfb007e82c33d, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.398687 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.405085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bd5a3-35a7-4198-a294-347d100eb18a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.410228 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.410967 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.411312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.411658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] PATH_INFO: `/v3/projects/41d58f9b1b6d45679985c525322f3223` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.412888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d58f9b1b6d45679985c525322f3223)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.415056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.416767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1304/2590] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:15 2022] PUT /identity/v3/projects/3727f950e6b0419292b928912891526a/users/03a483f7a1474b25b2ecfb007e82c33d/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:15.417528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.449786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:15.480161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d58f9b1b6d45679985c525322f3223', 'request_id': 'req-303a8e1b-5944-4548-87ff-6daed0a48621'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:15.486365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d58f9b1b6d45679985c525322f3223', 'request_id': 'req-303a8e1b-5944-4548-87ff-6daed0a48621'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:15.487633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34442cfa-61dc-485b-902a-6ccf8e1cba6a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.497299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d58f9b1b6d45679985c525322f3223', 'request_id': 'req-303a8e1b-5944-4548-87ff-6daed0a48621'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:15.517086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] The token cache is being invalidate because project 41d58f9b1b6d45679985c525322f3223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:15.517401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:15.518392 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:15.520156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1287/2591] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:15 2022] DELETE /identity/v3/projects/41d58f9b1b6d45679985c525322f3223 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:15.520772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.522125 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.570786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.572177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.572514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.572800 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] PATH_INFO: `/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd/users/987001aac6a04044a176dbba2ba5703b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.624111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86572fbdf3b47f79be6d5c3c28371fd, user_id=987001aac6a04044a176dbba2ba5703b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.631608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599311c5-e433-4823-86d0-41872c9321e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.633273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a8e1b-5944-4548-87ff-6daed0a48621 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:15.639829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:15.640692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.641130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.641509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.644065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.653590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1305/2592] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:15 2022] PUT /identity/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd/users/987001aac6a04044a176dbba2ba5703b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:15.662585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.663099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.663680 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.737232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] MFA Rules not processed for user `03a483f7a1474b25b2ecfb007e82c33d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:15.739288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.756465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:15.758199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2164b-3953-4632-b66d-290ca295555c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:15.770390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1288/2593] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:15.771931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:15.772490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:15.772928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:15.803627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:15.804864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.826339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:15.887477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1306/2594] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:15.888502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.890499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.917742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1289/2595] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:15.918622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:15.919954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:15.936139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:16.021682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e1a2c-a048-4717-a6a4-35adbd9c2865 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.027466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.028538 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.029152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.030647 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.031719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.033870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b8f7b-50ff-4d91-900f-d37030a7d217 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.053097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23d23e-cd90-4473-aaa0-fc201ea23cf8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.056986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1307/2596] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:16.058749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.059199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.059579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.059830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.060681 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.061236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.061457 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.077217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.079765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.097196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:16.098530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1290/2597] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:16.101640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.106159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:16.115194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae6310-50f2-43b9-b93b-725632456f50 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.120485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1308/2598] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:16.125293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.125708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.126484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.127874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.127874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] PATH_INFO: `/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd/users/987001aac6a04044a176dbba2ba5703b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.130102 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86572fbdf3b47f79be6d5c3c28371fd, user_id=987001aac6a04044a176dbba2ba5703b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.134170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.137387 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c4d7f-fd50-4459-86d7-38e897ca14c3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.146236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.146581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.147047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.147332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.148861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:16.149178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.152097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.157665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1291/2599] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:16 2022] PUT /identity/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd/users/987001aac6a04044a176dbba2ba5703b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:16.179413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1309/2600] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:16.180154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.277964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35713-c50c-4524-a19f-fb764cadd880 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.286319 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.287251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.287727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.288251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.289572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.292245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.371637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.378741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1292/2601] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:16 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:16.388695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784cc52-bc49-4a31-b5d4-a358d67aab59 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.474776 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.475590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.475920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.476262 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.476886 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.479092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.516395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1310/2602] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:16 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:16.520488 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.533262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb636c-b8fc-439d-88b7-dad53f119f31 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.540381 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.541001 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.541273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.541763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.542874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.545010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.558031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1293/2603] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:16.562715 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.574912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e7e1f-d6f8-485d-bf1b-4c14cfeea658 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.583840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.584491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.584768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.585035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.588992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.594221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b221476-711b-4981-bd1d-8c6306b133cc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.613793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1311/2604] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:16.617154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.634059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cd045-3bf6-4003-8019-0d8de8c18a06 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.644436 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.645229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.645753 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.646137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] PATH_INFO: `/v3/projects/72051b8bfe26418d92779884f3328814/users/ef468a0974b249e6ba0feb9fa94a2e09/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.671700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72051b8bfe26418d92779884f3328814, user_id=ef468a0974b249e6ba0feb9fa94a2e09, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:16.674466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:16.705251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1294/2605] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:16 2022] PUT /identity/v3/projects/72051b8bfe26418d92779884f3328814/users/ef468a0974b249e6ba0feb9fa94a2e09/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:16.707713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.708166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.708516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.770126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] MFA Rules not processed for user `ef468a0974b249e6ba0feb9fa94a2e09`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:16.772258 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:16.803750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:16.898952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.902038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1312/2606] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:16.926546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:16.964764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:16.989326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d3f7e-07a9-430d-bca9-75964a29f994 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.996297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9b48-6a50-4a54-bfec-39c24c58ec54 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:16.996874 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:16.997522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:16.998038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:16.998204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:16.998977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.000742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.003227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.003938 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.004282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.005036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.007428 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:17.039171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1295/2607] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:16 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:17.042891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.080981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.082569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.114002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1313/2608] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:30:17.176352 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29cee8-1fbb-43c1-aae0-1f3d00de5118 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.187012 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.187789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.188210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.188726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.190657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.192830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.223246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1296/2609] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:17 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:17.227849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.239977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df708ea5-ca87-4809-9c3f-b41edff05040 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.247461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.248145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.248425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.248699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.249368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.251322 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.269544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.280427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1314/2610] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:17.286249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308373ba-9241-4e21-abdf-d6b0acd99c50 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.292288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.292970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.293269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.293527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] PATH_INFO: `/v3/projects/1d3f32ea759c479cac370c74e9683982/users/28f76f64702e4800b645a4994054be7b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.311974 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3f32ea759c479cac370c74e9683982, user_id=28f76f64702e4800b645a4994054be7b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.314803 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.335760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1297/2611] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:17 2022] PUT /identity/v3/projects/1d3f32ea759c479cac370c74e9683982/users/28f76f64702e4800b645a4994054be7b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:17.340865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.364035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09b28-a6e7-4687-bd75-e79b7024c695 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.501217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.501945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.502232 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.502503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.511636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.520944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2a34f-99e1-45ec-a062-38c38c523327 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.529312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:17.556155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1315/2612] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:17.558522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.579658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9240ee-4276-4e24-b9c0-d07bd5811a2a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.589002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.589864 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.591018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.591018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] PATH_INFO: `/v3/projects/1d3f32ea759c479cac370c74e9683982/users/28f76f64702e4800b645a4994054be7b/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.594960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3f32ea759c479cac370c74e9683982, user_id=28f76f64702e4800b645a4994054be7b, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:17.600041 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:17.628235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1298/2613] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:17 2022] PUT /identity/v3/projects/1d3f32ea759c479cac370c74e9683982/users/28f76f64702e4800b645a4994054be7b/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:17.641164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42936c4-f2aa-4cf6-a46c-2e8a1aa06bd9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.641482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42936c4-f2aa-4cf6-a46c-2e8a1aa06bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.641736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42936c4-f2aa-4cf6-a46c-2e8a1aa06bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:17.757713 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f42936c4-f2aa-4cf6-a46c-2e8a1aa06bd9 None None] MFA Rules not processed for user `28f76f64702e4800b645a4994054be7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:17.761482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f42936c4-f2aa-4cf6-a46c-2e8a1aa06bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:17.867970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1316/2614] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:17.880160 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:17.950250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435b2bf-a253-40ab-aeb6-2c7f8792dd6a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:17.955509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:17.956345 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:17.956773 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:17.957225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:18.007509 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:18.037992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:18.039450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:18.049485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1299/2615] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:21.478261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b017-4941-41c1-9287-46f8366f3ac3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:21.478797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b017-4941-41c1-9287-46f8366f3ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:21.479111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b017-4941-41c1-9287-46f8366f3ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:21.510374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c257b017-4941-41c1-9287-46f8366f3ac3 None None] MFA Rules not processed for user `987001aac6a04044a176dbba2ba5703b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:21.511639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c257b017-4941-41c1-9287-46f8366f3ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:21.580961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1317/2616] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:21.597401 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:21.599313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:21.676646 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744a049-bdec-468d-a24f-04728343ff26 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:21.685261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:21.685261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:21.685261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:21.685261 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:21.685932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:21.813084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:21.814722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:21.830207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1300/2617] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:30:22.977158 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804e72-5f7e-4492-a473-864519ec0f70 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:22.977659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804e72-5f7e-4492-a473-864519ec0f70 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:22.977754 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804e72-5f7e-4492-a473-864519ec0f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:22.997974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8d804e72-5f7e-4492-a473-864519ec0f70 None None] MFA Rules not processed for user `3e7526ef9eb0426a855f5b2b9be06a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:22.999208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8d804e72-5f7e-4492-a473-864519ec0f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:23.006357 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1318/2618] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:25.056184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:25.132657 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bef86-307e-4236-b786-e5aa83e3de7d service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:25.137413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:25.138105 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:25.138385 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:25.138659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] PATH_INFO: `/v3/users/ef468a0974b249e6ba0feb9fa94a2e09` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:25.139799 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef468a0974b249e6ba0feb9fa94a2e09)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:25.141832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:25.156962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:25.195080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef468a0974b249e6ba0feb9fa94a2e09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:25.195998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef468a0974b249e6ba0feb9fa94a2e09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:25.196610 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef468a0974b249e6ba0feb9fa94a2e09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:25.209439 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef468a0974b249e6ba0feb9fa94a2e09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:25.218840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef468a0974b249e6ba0feb9fa94a2e09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:25.252036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1301/2619] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:25 2022] DELETE /identity/v3/users/ef468a0974b249e6ba0feb9fa94a2e09 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:25.258028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:25.258670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:25.259255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:25.291576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:25.294208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:25.366569 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1319/2620] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:25.377523 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:25.379579 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:25.453349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed65605-94d4-4e8e-904a-34b922055fbb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:25.457214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:25.457883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:25.458158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:25.458558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:25.468914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:25.470513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:25.483183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1302/2621] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:26.021269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:26.022945 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:26.041026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:26.054072 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0376b-f654-4819-b273-24c3842e7858 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:26.064206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:26.065122 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:26.065761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:26.066654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] PATH_INFO: `/v3/projects/72051b8bfe26418d92779884f3328814` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:26.068161 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72051b8bfe26418d92779884f3328814)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:26.071165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:26.183255 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72051b8bfe26418d92779884f3328814', 'request_id': 'req-1d4c4818-25b1-483c-879a-686defb5e323'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.202598 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72051b8bfe26418d92779884f3328814', 'request_id': 'req-1d4c4818-25b1-483c-879a-686defb5e323'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.211318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72051b8bfe26418d92779884f3328814', 'request_id': 'req-1d4c4818-25b1-483c-879a-686defb5e323'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.237883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] The token cache is being invalidate because project 72051b8bfe26418d92779884f3328814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:26.238429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.239373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.241409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1320/2622] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:26 2022] DELETE /identity/v3/projects/72051b8bfe26418d92779884f3328814 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:26.245573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:26.259648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a881-eb24-4954-8d35-b877cb83cc01 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:26.368776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:26.369847 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:26.370508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:26.371047 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] PATH_INFO: `/v3/users/28f76f64702e4800b645a4994054be7b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:26.373058 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f76f64702e4800b645a4994054be7b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:26.375468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:26.428855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:26.441909 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f76f64702e4800b645a4994054be7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.442639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f76f64702e4800b645a4994054be7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.443262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f76f64702e4800b645a4994054be7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.456765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f76f64702e4800b645a4994054be7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.466939 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c84ba235-cc95-4054-a3d2-ad0c56eddaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f76f64702e4800b645a4994054be7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.483682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1303/2623] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:26 2022] DELETE /identity/v3/users/28f76f64702e4800b645a4994054be7b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:26.776963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:26.778555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:26.795943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c4818-25b1-483c-879a-686defb5e323 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:26.860301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:26.860944 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:26.861227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:26.861501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] PATH_INFO: `/v3/projects/1d3f32ea759c479cac370c74e9683982` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:26.862658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3f32ea759c479cac370c74e9683982)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:26.864562 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:26.914240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3f32ea759c479cac370c74e9683982', 'request_id': 'req-83674e6c-e176-4e96-a27a-b91ef772dce6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.923002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3f32ea759c479cac370c74e9683982', 'request_id': 'req-83674e6c-e176-4e96-a27a-b91ef772dce6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.929210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3f32ea759c479cac370c74e9683982', 'request_id': 'req-83674e6c-e176-4e96-a27a-b91ef772dce6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.941834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] The token cache is being invalidate because project 1d3f32ea759c479cac370c74e9683982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:26.941834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.942431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-83674e6c-e176-4e96-a27a-b91ef772dce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:26.943924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1321/2624] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:26 2022] DELETE /identity/v3/projects/1d3f32ea759c479cac370c74e9683982 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:26.948829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:26.949382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:26.949996 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:26.971487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:26.972724 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:27.035710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1304/2625] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:27.052558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.052558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.052558 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.084140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:27.085465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:27.095642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1322/2626] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:27.099788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.111143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e96292-ef96-4e29-a6ca-656b0ad79733 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.116408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.117110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.118670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.118670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.118670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:27.118670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.120432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.142120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1305/2627] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:30:27.143478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.155957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeebbc-0395-420d-8e43-5df620ee9f4c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.164883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.166216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.167120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.167730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.168549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.171172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.178023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:27.183857 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.185869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:27.219773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1323/2628] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:27 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:27.221132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.280992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585016af-ea73-43e2-8a4b-39cfbe4bb1a2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.348315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e267f-28c6-4152-beb0-e7a5c3213d60 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.357435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.358133 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.358413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.358685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.359376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.361395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.390786 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.391693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.395227 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.395393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1324/2629] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:27 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:27.395645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.396155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.397793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:27.412602 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b970d-96f7-4cc5-9457-6e41d0bd4a34 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.412807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:27.417793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.418411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.418695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.419005 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.420240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.422043 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.440119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1325/2630] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:27.443503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.469041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a698f3-f0da-4cf4-97cc-36e31d991522 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.474743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.475404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.476058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.476058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.476684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.478356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1535d34-034b-40c2-bcdc-7c70e6d6f30d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.503574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.505045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.520834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1306/2631] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:30:27.642087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1326/2632] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:27.646510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:27.666946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc497ae6-6991-4acc-890b-1c9308c44254 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:27.676313 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:27.678199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.678569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.678845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] PATH_INFO: `/v3/projects/37e678f99a2945c0885d266e9e8f5453/users/b2eab9d14fe543b29fa0c3de9fb5a922/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.710315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e678f99a2945c0885d266e9e8f5453, user_id=b2eab9d14fe543b29fa0c3de9fb5a922, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:27.713906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:27.741651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1307/2633] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:27 2022] PUT /identity/v3/projects/37e678f99a2945c0885d266e9e8f5453/users/b2eab9d14fe543b29fa0c3de9fb5a922/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:27.753078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:27.753451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:27.753825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:27.802554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] MFA Rules not processed for user `b2eab9d14fe543b29fa0c3de9fb5a922`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:27.803340 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:27.919793 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1327/2634] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:27.924761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.016091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9399e5-0154-48f1-8994-d362b4bf9e6a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.021312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.021954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.022239 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.022514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.023140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.025121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.054364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1308/2635] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:27 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:28.058606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.069950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d98cb-154c-45a7-85ce-21e506416bcf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.113669 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:28.131672 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.132206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.132484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.132757 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.133771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.135710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.160025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1328/2636] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:28 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:28.164133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.176017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3531a1-563d-49f6-a0ca-0fc9fa76ed39 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.180887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.181487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.181801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.182074 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.182711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.184340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.195910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1309/2637] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:28.200210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.211352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285a75b-0e3a-4cf9-ae1d-a6c5584e69e1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.216716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.217329 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.217634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.217917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.218561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.220155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc9cd6-7484-4d85-afc2-c3beca98dfec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.231561 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1329/2638] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:28.235787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.248679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93014-37d1-45a6-aaa7-de906e8c9228 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.254926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.255588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.255868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.256138 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] PATH_INFO: `/v3/projects/aaa48a9e676c45cf9359a883da3b29cb/users/80dc83f119014aaf9550acec73682cfe/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.272339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa48a9e676c45cf9359a883da3b29cb, user_id=80dc83f119014aaf9550acec73682cfe, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.275217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.291005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1310/2639] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:28 2022] PUT /identity/v3/projects/aaa48a9e676c45cf9359a883da3b29cb/users/80dc83f119014aaf9550acec73682cfe/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:28.302675 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.302960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.303233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.332298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] MFA Rules not processed for user `80dc83f119014aaf9550acec73682cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:28.333502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:28.413233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1330/2640] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:28.414572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.455377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:28.481566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ac005-0c48-41a7-8ed8-4c48bafe5025 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.486230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.486837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.487108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.487376 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.487993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.489934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.526277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1311/2641] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:28 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:28.531138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.542099 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a169fcf-f623-4346-a233-70d310f2ed87 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.601286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.602049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.602334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.602614 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.603250 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.605222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.626676 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1331/2642] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:28 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:28.631029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.645399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026b887-ff9f-4931-843c-8c1c41658cb2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.650380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.650993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.651271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.651542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.652184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.653859 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.666906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1312/2643] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:28.669906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.680462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d012d04-a09c-4fc1-ab25-1cf6e4e7c316 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.686461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.687183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.687465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.687742 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] PATH_INFO: `/v3/projects/7afdb3a1889a43e28c664a442f16f591/users/33b1bc5113fc4937ae7df3f945aa2169/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.705641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afdb3a1889a43e28c664a442f16f591, user_id=33b1bc5113fc4937ae7df3f945aa2169, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.708345 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.727749 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1332/2644] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:28 2022] PUT /identity/v3/projects/7afdb3a1889a43e28c664a442f16f591/users/33b1bc5113fc4937ae7df3f945aa2169/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:28.731848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.742606 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad88024-a737-4698-88ef-329eedfd5eae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.802538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.803154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.803434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.803706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.804344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.806009 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2c136-306c-4ed0-bf96-fe307938a42f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.819674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1313/2645] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:28.823677 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:28.837103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9251741-9f55-4e48-9f82-9f144425a095 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:28.842306 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:28.842924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.843199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.843466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] PATH_INFO: `/v3/projects/7afdb3a1889a43e28c664a442f16f591/users/33b1bc5113fc4937ae7df3f945aa2169/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.845732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afdb3a1889a43e28c664a442f16f591, user_id=33b1bc5113fc4937ae7df3f945aa2169, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:28.848442 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:28.865829 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1333/2646] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:28 2022] PUT /identity/v3/projects/7afdb3a1889a43e28c664a442f16f591/users/33b1bc5113fc4937ae7df3f945aa2169/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:28.870101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:28.870486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:28.870769 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:28.902271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] MFA Rules not processed for user `33b1bc5113fc4937ae7df3f945aa2169`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:28.903546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:28.996876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1314/2647] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:29.015485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:29.019204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:29.029879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:29.100805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb129e-de1d-4fee-a2d8-0db49819efe1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:29.104887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:29.105507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:29.105850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:29.106109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:29.195313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:29.196763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:29.208086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1334/2648] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:32.132475 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:32.135298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:32.138063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:32.151674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78332a68-7a4c-43d0-b2b3-d6b4106e79b9 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:32.155809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:32.156650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:32.157016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:32.157521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:32.194254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:32.253530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c17d-ae93-4018-a148-1b8cadca4a54 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:32.268708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:32.270950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77406b-fc67-4eea-81db-60594f8135e2 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:32.274337 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:32.275480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:32.276236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:32.276736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] PATH_INFO: `/v3/users/4c4b11da98284bb6a4cbc41e993cb10a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:32.278583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4b11da98284bb6a4cbc41e993cb10a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:32.281587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:32.283064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1335/2649] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:32.382459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4b11da98284bb6a4cbc41e993cb10a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:32.383252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4b11da98284bb6a4cbc41e993cb10a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:32.384780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4b11da98284bb6a4cbc41e993cb10a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:32.403340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4b11da98284bb6a4cbc41e993cb10a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:32.432330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4b11da98284bb6a4cbc41e993cb10a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:32.454309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1315/2650] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:32 2022] DELETE /identity/v3/users/4c4b11da98284bb6a4cbc41e993cb10a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:32.460177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:32.460464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:32.460740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:32.490175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:32.491685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:32.563192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:32.576412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1336/2651] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:32.588161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:32.590542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:32.671679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80fc7e-58b9-431a-afb1-455c4d691573 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:32.676190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:32.676877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:32.677225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:32.677502 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:32.689057 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:32.690505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:32.703140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1316/2652] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:33.016544 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:33.019062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:33.060744 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:33.062719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:33.079866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac6d5-0935-4cfc-93f6-7afc15c6b03e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:33.086154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:33.088794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:33.089086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:33.089359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] PATH_INFO: `/v3/projects/17f0779cc2d34ff9a012b0d04361d514` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:33.098760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f0779cc2d34ff9a012b0d04361d514)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:33.100692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:33.130802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134f57-f68a-4b9f-9ad4-313ec8547a5e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:33.137632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:33.137632 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:33.138462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:33.138981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:33.140780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:33.174130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:33.175583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f0779cc2d34ff9a012b0d04361d514', 'request_id': 'req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.183326 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f0779cc2d34ff9a012b0d04361d514', 'request_id': 'req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.193441 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f0779cc2d34ff9a012b0d04361d514', 'request_id': 'req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.208221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] The token cache is being invalidate because project 17f0779cc2d34ff9a012b0d04361d514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:33.208512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.209329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.211573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1317/2653] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:33 2022] DELETE /identity/v3/projects/17f0779cc2d34ff9a012b0d04361d514 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:33.216056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:33.216640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:33.217789 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:33.218135 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:33.228574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1337/2654] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:33.312190 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b54d8-d5ac-4c86-9c07-e5aa7217beff admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:33.318499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:33.319187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:33.319524 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:33.319798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] PATH_INFO: `/v3/users/60df787466da4d95a3d9d85c6ccc128b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:33.321027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60df787466da4d95a3d9d85c6ccc128b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:33.323021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:33.399617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60df787466da4d95a3d9d85c6ccc128b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.400518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60df787466da4d95a3d9d85c6ccc128b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.401902 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60df787466da4d95a3d9d85c6ccc128b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.423118 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60df787466da4d95a3d9d85c6ccc128b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.432183 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60df787466da4d95a3d9d85c6ccc128b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.456715 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1318/2655] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:33 2022] DELETE /identity/v3/users/60df787466da4d95a3d9d85c6ccc128b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:33.797833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:33.815382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e99a4-ddb0-474c-837e-1040ee097bf8 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:33.887181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:33.887844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:33.888124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:33.888395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] PATH_INFO: `/v3/projects/52af5ef28ca347c0a68b3a2736278409` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:33.889514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] RBAC: Authorizing `identity:delete_project(project_id=52af5ef28ca347c0a68b3a2736278409)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:33.891495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:33.944297 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52af5ef28ca347c0a68b3a2736278409', 'request_id': 'req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.946020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:33.954452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52af5ef28ca347c0a68b3a2736278409', 'request_id': 'req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.964540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52af5ef28ca347c0a68b3a2736278409', 'request_id': 'req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.977678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] The token cache is being invalidate because project 52af5ef28ca347c0a68b3a2736278409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:33.978209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.978883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:33.980395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1338/2656] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:33 2022] DELETE /identity/v3/projects/52af5ef28ca347c0a68b3a2736278409 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:33.985473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:33.988408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:34.002110 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a040-2323-44e2-8aba-1b9fe9d6fe6e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:34.061500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:34.070630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:34.071234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:34.071710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:34.072123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] PATH_INFO: `/v3/users/03a483f7a1474b25b2ecfb007e82c33d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:34.073360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a483f7a1474b25b2ecfb007e82c33d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:34.075622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:34.146743 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a483f7a1474b25b2ecfb007e82c33d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.147588 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a483f7a1474b25b2ecfb007e82c33d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.148221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a483f7a1474b25b2ecfb007e82c33d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.163010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a483f7a1474b25b2ecfb007e82c33d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.175394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6883507d-2682-4afc-9c6f-44f7477a46bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a483f7a1474b25b2ecfb007e82c33d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.193266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1319/2657] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:33 2022] DELETE /identity/v3/users/03a483f7a1474b25b2ecfb007e82c33d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:34.520927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:34.522482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:34.545582 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a85d9c-e484-4054-a0fa-67ce3bb7aaea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:34.634553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:34.636109 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:34.636391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:34.636666 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] PATH_INFO: `/v3/projects/3727f950e6b0419292b928912891526a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:34.638466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3727f950e6b0419292b928912891526a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:34.640542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:34.717274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3727f950e6b0419292b928912891526a', 'request_id': 'req-a55fcae6-d300-4564-830c-1e4c634e993d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.738417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3727f950e6b0419292b928912891526a', 'request_id': 'req-a55fcae6-d300-4564-830c-1e4c634e993d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.766124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3727f950e6b0419292b928912891526a', 'request_id': 'req-a55fcae6-d300-4564-830c-1e4c634e993d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.816658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] The token cache is being invalidate because project 3727f950e6b0419292b928912891526a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:34.816960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.820581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a55fcae6-d300-4564-830c-1e4c634e993d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:34.826292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1339/2658] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:34 2022] DELETE /identity/v3/projects/3727f950e6b0419292b928912891526a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:34.832237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:34.832532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:34.832808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:34.873433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:34.874912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:34.961640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1320/2659] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:34.970157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c31909-71de-41ef-b732-2e42327147fe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:34.970419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c31909-71de-41ef-b732-2e42327147fe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:34.972342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c31909-71de-41ef-b732-2e42327147fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:34.994431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-17c31909-71de-41ef-b732-2e42327147fe None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:34.996034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-17c31909-71de-41ef-b732-2e42327147fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:35.005911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1340/2660] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:35.010364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.021739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b667fdf-f1e5-42e4-9af3-7b2be50e1bd5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.029365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.030356 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.030684 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.030952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.031600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:35.031898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.033565 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.047048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1321/2661] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:35.051432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c31909-71de-41ef-b732-2e42327147fe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.061963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c31909-71de-41ef-b732-2e42327147fe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.066983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.067670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.067953 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.068230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.068866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.070830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.099238 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1341/2662] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:35 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:35.103371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.119334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744191fd-7087-4283-82c7-7dc88d17f73a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.191472 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.192145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.192293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.192570 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.193206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.195232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.204591 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:35.221912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1322/2663] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:35 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:35.223096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.235028 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bc9db-c215-4b41-92a2-edb38c311873 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.240147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.240821 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.241203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.241545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.242290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.244082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.256689 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1342/2664] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:35.260958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.276470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7641-f957-4efa-bca8-8fe1d38d8093 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.282328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.282934 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.283218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.283495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.284217 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.285942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec7a03-c847-4c4f-a031-677e7befff0c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.303232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1323/2665] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:35.304942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.317903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1f480-ffba-4130-9aff-6cc91a277ae8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.324785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.325688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.326126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.326403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] PATH_INFO: `/v3/projects/bff6c4d43a4743a39b3843069bf6c1dd/users/7c846055d75d4ce785d30f9bd3af8c93/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.345794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff6c4d43a4743a39b3843069bf6c1dd, user_id=7c846055d75d4ce785d30f9bd3af8c93, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.349337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.364060 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1343/2666] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:35 2022] PUT /identity/v3/projects/bff6c4d43a4743a39b3843069bf6c1dd/users/7c846055d75d4ce785d30f9bd3af8c93/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:35.376522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.376810 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.377088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.405354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] MFA Rules not processed for user `7c846055d75d4ce785d30f9bd3af8c93`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:35.406617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:35.489733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1324/2667] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:35.503869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.506965 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:35.515038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:35.582755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c36ad4-87e5-40be-993d-426dd0028457 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.586839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.587450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.587727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.587996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.597578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.599024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d0bba-c6f5-4c30-8dff-7f59ec81ea6e service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.608417 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1344/2668] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:35.629435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.697133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af681bc-4e4d-42f4-9747-173368a8d9ea None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:35.702447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:35.703085 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.703361 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.703636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] PATH_INFO: `/v3/users/7c846055d75d4ce785d30f9bd3af8c93` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.704770 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c846055d75d4ce785d30f9bd3af8c93)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:35.706889 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:35.764368 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c846055d75d4ce785d30f9bd3af8c93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:35.764943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c846055d75d4ce785d30f9bd3af8c93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:35.765558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c846055d75d4ce785d30f9bd3af8c93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:35.779363 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c846055d75d4ce785d30f9bd3af8c93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:35.784692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:35.787171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c846055d75d4ce785d30f9bd3af8c93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:35.801722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1325/2669] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:35 2022] DELETE /identity/v3/users/7c846055d75d4ce785d30f9bd3af8c93 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:35.806471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:35.806767 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:35.807189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:35.837127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:35.838446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:35.910856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1345/2670] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:35.921820 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:35.923312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.034341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344c26c-61af-49fb-a098-3ba37123ae36 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:36.045752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:36.046380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.046671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.046945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.060995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:36.064164 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac327df-45ae-420a-837a-69baba7bd2d6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:36.079811 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1326/2671] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:36.139100 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:36.144023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.163330 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7448edc-aad2-42e6-9bad-6b642faef71d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:36.171802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:36.172426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.172708 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.172977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] PATH_INFO: `/v3/users/9b43bd85a0fb429c9af5ef8de7ee818e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.174261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b43bd85a0fb429c9af5ef8de7ee818e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:36.176191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:36.278014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b43bd85a0fb429c9af5ef8de7ee818e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.287756 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b43bd85a0fb429c9af5ef8de7ee818e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.314309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b43bd85a0fb429c9af5ef8de7ee818e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.325746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b43bd85a0fb429c9af5ef8de7ee818e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.326645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b43bd85a0fb429c9af5ef8de7ee818e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.340539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1346/2672] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:36 2022] DELETE /identity/v3/users/9b43bd85a0fb429c9af5ef8de7ee818e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:36.343637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.344210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.344210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.368369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:36.370145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.493384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1327/2673] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:36.501098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:36.502645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.518695 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:36.596230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061d44e-036e-431f-a74a-f4453856433d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:36.601222 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:36.602275 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.602743 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.603026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.619377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:36.620929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c9797-455f-4b7c-a9a4-aa03feda0931 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:36.635290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1347/2674] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:36.732994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:36.735238 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.747775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee55e6-0330-41df-a89b-7b9963f542c2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:36.760998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:36.761517 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.761867 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.762159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] PATH_INFO: `/v3/projects/bff6c4d43a4743a39b3843069bf6c1dd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.766761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff6c4d43a4743a39b3843069bf6c1dd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:36.767808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:36.820950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff6c4d43a4743a39b3843069bf6c1dd', 'request_id': 'req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.833536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff6c4d43a4743a39b3843069bf6c1dd', 'request_id': 'req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.861224 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff6c4d43a4743a39b3843069bf6c1dd', 'request_id': 'req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.875723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] The token cache is being invalidate because project bff6c4d43a4743a39b3843069bf6c1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:36.876018 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.876668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:36.878206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1328/2675] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:36 2022] DELETE /identity/v3/projects/bff6c4d43a4743a39b3843069bf6c1dd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:36.883288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9deb-72b4-4bd5-bee9-45f271a4e5ef None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.883581 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9deb-72b4-4bd5-bee9-45f271a4e5ef None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.883861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9deb-72b4-4bd5-bee9-45f271a4e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.903722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f67b9deb-72b4-4bd5-bee9-45f271a4e5ef None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:36.904947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f67b9deb-72b4-4bd5-bee9-45f271a4e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.968814 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1348/2676] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:36.979904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:36.980131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:36.980395 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:36.980730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:36.981897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:36.999908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc10c6-b1ea-4c68-a185-1c5136a94ab5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.002476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.003794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.007705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.008460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.008764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.009071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] PATH_INFO: `/v3/projects/bfeeb93d1df94eb38ee5fb54d5c459de` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.010374 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfeeb93d1df94eb38ee5fb54d5c459de)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.012486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.018149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1349/2677] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:37.022398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.026318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:37.032835 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c0a9e-70dc-44b4-97c3-8d9b32fad9fb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.038082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.038725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.039006 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.039276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.039954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:37.040221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.041823 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.057768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1350/2678] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:30:37.058912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.067459 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfeeb93d1df94eb38ee5fb54d5c459de', 'request_id': 'req-769fdeb9-d2dc-4ead-8461-9424d79590d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.069897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c566a5-f196-4077-8421-36c7a4470420 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.073039 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfeeb93d1df94eb38ee5fb54d5c459de', 'request_id': 'req-769fdeb9-d2dc-4ead-8461-9424d79590d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.082750 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfeeb93d1df94eb38ee5fb54d5c459de', 'request_id': 'req-769fdeb9-d2dc-4ead-8461-9424d79590d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.094659 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] The token cache is being invalidate because project bfeeb93d1df94eb38ee5fb54d5c459de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:37.094958 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.095653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-769fdeb9-d2dc-4ead-8461-9424d79590d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.097157 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1329/2679] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:36 2022] DELETE /identity/v3/projects/bfeeb93d1df94eb38ee5fb54d5c459de => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:37.102890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce153562-48f9-4baa-a3ae-72198315b81f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.103178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce153562-48f9-4baa-a3ae-72198315b81f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.103464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce153562-48f9-4baa-a3ae-72198315b81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.123471 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ce153562-48f9-4baa-a3ae-72198315b81f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.124692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce153562-48f9-4baa-a3ae-72198315b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.129932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.130794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.131077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.131353 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.131974 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.133909 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.174983 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1351/2680] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:37.176061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.177832 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.194976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1330/2681] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:37.198449 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:37.203375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.203666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.203943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.225935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.227134 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.244967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d3a-0f18-48be-ad35-445e7154052f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.250827 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.251443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.251758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.252036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.252684 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.254716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.279074 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1352/2682] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:37.282964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.289522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1331/2683] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:37.292615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c5e6f-0848-4b85-b6cb-7cfd087a8210 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.293662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.297926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.298755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.299041 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.299911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.300566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.302361 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.302989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41be7c5-aa44-4c97-9092-84149900f750 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.307995 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.308618 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.309038 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.309355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.310617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:37.310952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.312849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.313519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1353/2684] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:30:37.317656 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.327882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1332/2685] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 22:30:37.328049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804ae7c-c26c-4387-a9eb-ff238b748537 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.328779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.332085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.332696 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.332999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.333438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.333981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.335571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.338424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe5448-94e1-418d-8150-533c9eda8168 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.343921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.344656 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.344872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.345168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.345837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.347927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.357650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1354/2686] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:37.358777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.369645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602ac8c-d07e-4298-8fc8-3cbcacd8bf47 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.375694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.376351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.376636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.376922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] PATH_INFO: `/v3/projects/d93668b12728448aa781f26f4d7bbe4e/users/1d3dbb2ea33246179c533b46a45cdb93/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.382535 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1333/2687] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:37.383511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.389423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:37.392599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93668b12728448aa781f26f4d7bbe4e, user_id=1d3dbb2ea33246179c533b46a45cdb93, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.395388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff58d3-b457-4efc-9c8d-a6cd6b375a73 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.408805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1355/2688] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:37 2022] PUT /identity/v3/projects/d93668b12728448aa781f26f4d7bbe4e/users/1d3dbb2ea33246179c533b46a45cdb93/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:37.421251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.421674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.421947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.449466 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] MFA Rules not processed for user `1d3dbb2ea33246179c533b46a45cdb93`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.450473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682f8d1-4b3b-4a54-a640-2b78ecbea4a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.450865 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.541111 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.541766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.542049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.542330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.542956 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.545617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.555029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1356/2689] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:37.555720 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.557033 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.574125 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3b15-0a68-4e4f-a568-b74bf7425790 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.577452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1334/2690] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:30:37.578339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.580022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.582878 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.584204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.584789 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.585463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] PATH_INFO: `/v3/users/b2eab9d14fe543b29fa0c3de9fb5a922` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.587179 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2eab9d14fe543b29fa0c3de9fb5a922)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.591391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.636911 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:37.638910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2eab9d14fe543b29fa0c3de9fb5a922'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.643512 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f352aa-3d08-4433-904a-92a63de987fd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.647221 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2eab9d14fe543b29fa0c3de9fb5a922'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.647805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.648418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.648699 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.648973 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.658031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.659474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.660235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2eab9d14fe543b29fa0c3de9fb5a922'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.668700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1335/2691] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:37.668959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2eab9d14fe543b29fa0c3de9fb5a922'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.670128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.670264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d12c1fde-74e8-43e2-92ae-85cc9f1e2789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2eab9d14fe543b29fa0c3de9fb5a922'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:37.677077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1357/2692] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:37 2022] DELETE /identity/v3/users/b2eab9d14fe543b29fa0c3de9fb5a922 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:37.680559 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9e8ac-20e0-484a-b188-6d5f031c548c service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.681506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.681928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.682159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.709327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.710705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.739580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:37.757338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.757994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.758271 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.758542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.759182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.760976 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.773027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1336/2693] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:37.777348 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.781174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1358/2694] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:37.784779 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.791296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.793516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:37.807095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db3c80-04ae-4934-9ddf-0950f7902f63 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.818036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.818954 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.819658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.819658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.820088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.822071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.836892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1337/2695] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:37.842073 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:37.859776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc1c2-53d2-49a6-9f8b-93113cca2c7c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.865341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.866049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.866388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.866661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] PATH_INFO: `/v3/projects/cb3054dcaa3e49598f8354def66953a7/users/dc1b4429caad4aa1bd8cdefa94134286/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.890221 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3054dcaa3e49598f8354def66953a7, user_id=dc1b4429caad4aa1bd8cdefa94134286, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.893194 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ed3fa-af56-4e59-8168-da3b8a9f47c6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.897024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2c083-f4a6-4d2d-9631-0df763b48a70 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:37.902042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:37.902824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.904324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.904324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.917870 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1338/2696] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:37 2022] PUT /identity/v3/projects/cb3054dcaa3e49598f8354def66953a7/users/dc1b4429caad4aa1bd8cdefa94134286/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:37.918123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:37.918729 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:37.925317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:37.925709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:37.926043 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:37.933775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1359/2697] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:37.963219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] MFA Rules not processed for user `dc1b4429caad4aa1bd8cdefa94134286`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:37.964800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:38.093825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1339/2698] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:38.102616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:38.104434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:38.209431 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:38.214062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c364c24-c1fe-44a8-8613-a3fe4675861c service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:38.281516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71ebfbeddd1943f6af2c0229edbbc41d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:38.283622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:38.283622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:38.283622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:38.292285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:38.293890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:38.296455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:38.298229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:38.304375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1360/2699] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:38.389495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09a8a-8528-43ed-8040-5ede93040aa5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:38.396584 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:38.397216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:38.397489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:38.397788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] PATH_INFO: `/v3/projects/37e678f99a2945c0885d266e9e8f5453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:38.398904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e678f99a2945c0885d266e9e8f5453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:38.401103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:38.441250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:38.452112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e678f99a2945c0885d266e9e8f5453', 'request_id': 'req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.458210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e678f99a2945c0885d266e9e8f5453', 'request_id': 'req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.466998 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e678f99a2945c0885d266e9e8f5453', 'request_id': 'req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.478378 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] The token cache is being invalidate because project 37e678f99a2945c0885d266e9e8f5453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:38.478671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.479316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.480892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1340/2700] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:38 2022] DELETE /identity/v3/projects/37e678f99a2945c0885d266e9e8f5453 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:38.486036 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:38.497025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940e35-7b8c-4af2-a314-8e98a002b447 service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:38.564385 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:38.565086 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:38.565292 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:38.565566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] PATH_INFO: `/v3/users/80dc83f119014aaf9550acec73682cfe` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:38.566942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80dc83f119014aaf9550acec73682cfe)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:38.568903 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:38.627164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80dc83f119014aaf9550acec73682cfe'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.635864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80dc83f119014aaf9550acec73682cfe'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.648871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80dc83f119014aaf9550acec73682cfe'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.656929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80dc83f119014aaf9550acec73682cfe'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.657553 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80dc83f119014aaf9550acec73682cfe'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:38.663942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1361/2701] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:38 2022] DELETE /identity/v3/users/80dc83f119014aaf9550acec73682cfe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:38.957299 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:38.958812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:38.972553 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ee7b-aadc-43d5-b5b0-2bfc80945c1d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:39.029903 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:39.030520 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.030798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.031089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] PATH_INFO: `/v3/projects/aaa48a9e676c45cf9359a883da3b29cb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.032329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa48a9e676c45cf9359a883da3b29cb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:39.034377 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:39.086287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa48a9e676c45cf9359a883da3b29cb', 'request_id': 'req-a313e14f-7bc5-4029-850c-7ff003a0b0bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.093154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa48a9e676c45cf9359a883da3b29cb', 'request_id': 'req-a313e14f-7bc5-4029-850c-7ff003a0b0bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.102193 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa48a9e676c45cf9359a883da3b29cb', 'request_id': 'req-a313e14f-7bc5-4029-850c-7ff003a0b0bd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.115080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] The token cache is being invalidate because project aaa48a9e676c45cf9359a883da3b29cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:39.115364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.116097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.117622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1341/2702] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:38 2022] DELETE /identity/v3/projects/aaa48a9e676c45cf9359a883da3b29cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:39.121686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:39.123595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:39.137320 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c90ca-e7f5-4cfa-a828-9f3e4602cf7d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:39.159910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:39.211899 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:39.212731 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.213090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.213388 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] PATH_INFO: `/v3/users/33b1bc5113fc4937ae7df3f945aa2169` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.214863 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b1bc5113fc4937ae7df3f945aa2169)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:39.217488 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:39.279830 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b1bc5113fc4937ae7df3f945aa2169'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.290034 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b1bc5113fc4937ae7df3f945aa2169'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.303781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b1bc5113fc4937ae7df3f945aa2169'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.314153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b1bc5113fc4937ae7df3f945aa2169'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.315414 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-977e163f-fd6e-4ecd-ab92-4ab271771307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b1bc5113fc4937ae7df3f945aa2169'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.327681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1362/2703] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:39 2022] DELETE /identity/v3/users/33b1bc5113fc4937ae7df3f945aa2169 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:39.610971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:39.613448 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:39.639324 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313e14f-7bc5-4029-850c-7ff003a0b0bd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:39.702293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:39.710254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:39.711318 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.711868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.712108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] PATH_INFO: `/v3/projects/7afdb3a1889a43e28c664a442f16f591` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.713804 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afdb3a1889a43e28c664a442f16f591)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:39.716307 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:39.763928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afdb3a1889a43e28c664a442f16f591', 'request_id': 'req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.770177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afdb3a1889a43e28c664a442f16f591', 'request_id': 'req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.778143 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afdb3a1889a43e28c664a442f16f591', 'request_id': 'req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.789083 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] The token cache is being invalidate because project 7afdb3a1889a43e28c664a442f16f591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:39.789367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.790023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c74f11c9-a2ad-4e5e-9d4c-901823cfcffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:39.791497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1342/2704] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:39 2022] DELETE /identity/v3/projects/7afdb3a1889a43e28c664a442f16f591 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:39.797023 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.797494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.797628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.818791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:39.820446 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:39.884186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1363/2705] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:39.892437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.892720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.892991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.912706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:39.914027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:39.923006 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1343/2706] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:39.927234 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:39.937138 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba9b5-e1ea-4fe7-84aa-1c85998dc7c8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:39.941701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:39.942296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.942571 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.942841 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.943480 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:39.943807 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:39.945403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:39.960287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1364/2707] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:39.965343 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:39.975862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08496b8-7a4f-4165-914a-58aaecdcf6c3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:39.980834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:39.982065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:39.982349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:39.982613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:39.983234 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:39.985122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.012226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1344/2708] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:39 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:40.016551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:40.026819 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01590d8-557e-442d-9091-5ce1e0169707 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:40.056645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:40.089265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:40.089850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.090134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.090411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.091572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:40.093471 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.115078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1365/2709] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:40 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:40.119495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:40.129884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72f43e-4d7d-4e80-844c-3dba5938f9bf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:40.134482 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:40.135312 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.135432 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.135616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.136250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:40.137913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.147697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1345/2710] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:40.151707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:40.162040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7385-3230-494e-a14d-be13c26e5545 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:40.166554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:40.167137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.167411 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.167681 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.168317 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:40.170531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb779782-0d21-4b34-854f-fa1ff74d1f97 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.180127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1366/2711] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:40.184100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:40.194815 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be1927-f4c0-447e-9414-ab391f704660 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:40.199952 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:40.201140 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.201415 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.201727 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] PATH_INFO: `/v3/projects/61268e79778c4a20b3b7beca375d5706/users/baa2e8e668894ea4b82fea3a2bc1fe9e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.217975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61268e79778c4a20b3b7beca375d5706, user_id=baa2e8e668894ea4b82fea3a2bc1fe9e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:40.220716 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.233548 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1346/2712] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:40 2022] PUT /identity/v3/projects/61268e79778c4a20b3b7beca375d5706/users/baa2e8e668894ea4b82fea3a2bc1fe9e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:40.238638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146320-1a6f-4a69-93e0-7c8d9908b5f0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.238925 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146320-1a6f-4a69-93e0-7c8d9908b5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.239252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146320-1a6f-4a69-93e0-7c8d9908b5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.257633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d6146320-1a6f-4a69-93e0-7c8d9908b5f0 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:40.258828 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6146320-1a6f-4a69-93e0-7c8d9908b5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:40.321413 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1367/2713] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:40.331569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:40.333266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:40.390328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004792-1f5d-43c1-a811-3b3ffdcb8e7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:40.394059 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:40.394661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:40.394935 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:40.395204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:40.404660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:40.406121 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:40.415477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1347/2714] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:41.039056 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767038e-6a7a-46ff-a834-952354f8efbc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:41.039554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767038e-6a7a-46ff-a834-952354f8efbc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:41.039646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767038e-6a7a-46ff-a834-952354f8efbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:41.069274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d767038e-6a7a-46ff-a834-952354f8efbc None None] MFA Rules not processed for user `baa2e8e668894ea4b82fea3a2bc1fe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:41.070542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d767038e-6a7a-46ff-a834-952354f8efbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:41.149038 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1368/2715] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:41.160489 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:41.162035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:41.191778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:41.228340 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9c7c-a8d9-44c2-b546-4327176f9dfc service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:41.232599 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:41.233325 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:41.233668 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:41.233907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:41.242543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:41.244010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:41.253622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1348/2716] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:44.718794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b542af-1bda-431f-918e-d5e971c1f637 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:44.719350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b542af-1bda-431f-918e-d5e971c1f637 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:44.719350 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b542af-1bda-431f-918e-d5e971c1f637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:44.738120 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-82b542af-1bda-431f-918e-d5e971c1f637 None None] MFA Rules not processed for user `baa2e8e668894ea4b82fea3a2bc1fe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:44.739327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-82b542af-1bda-431f-918e-d5e971c1f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:44.746592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1369/2717] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:47.329114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:47.330721 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:47.346596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e5427-0293-4c05-ad0b-e43e354852dd service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:47.362065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:47.363065 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:47.363551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:47.364017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] PATH_INFO: `/v3/users/1d3dbb2ea33246179c533b46a45cdb93` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:47.365900 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3dbb2ea33246179c533b46a45cdb93)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:47.368114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:47.429212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3dbb2ea33246179c533b46a45cdb93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:47.431388 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3dbb2ea33246179c533b46a45cdb93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:47.432521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3dbb2ea33246179c533b46a45cdb93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:47.450470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3dbb2ea33246179c533b46a45cdb93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:47.458355 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3dbb2ea33246179c533b46a45cdb93'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:47.472391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1349/2718] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:30:47 2022] DELETE /identity/v3/users/1d3dbb2ea33246179c533b46a45cdb93 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:47.477055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:47.477531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:47.477849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:47.498906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:47.500293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:47.536300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:47.583373 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1370/2719] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:47.596036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:47.597985 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:47.669367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255e2b9-6543-408c-96c4-c8a4879517b8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:47.674926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:47.675648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:47.675844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:47.676120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:47.688222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:47.689774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba15e9-5a5f-4b30-91da-63267ae45533 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:47.730988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1350/2720] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:48.371042 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:48.373364 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:48.391356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f67c-ba43-4e35-b647-a6733db048e9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:48.414963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:48.415732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:48.416110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:48.416694 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] PATH_INFO: `/v3/projects/d93668b12728448aa781f26f4d7bbe4e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:48.418541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93668b12728448aa781f26f4d7bbe4e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:48.421208 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:48.510804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93668b12728448aa781f26f4d7bbe4e', 'request_id': 'req-8f575fec-934d-43b8-8c8a-f85943783147'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:48.528469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93668b12728448aa781f26f4d7bbe4e', 'request_id': 'req-8f575fec-934d-43b8-8c8a-f85943783147'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:48.545476 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93668b12728448aa781f26f4d7bbe4e', 'request_id': 'req-8f575fec-934d-43b8-8c8a-f85943783147'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:48.571668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] The token cache is being invalidate because project d93668b12728448aa781f26f4d7bbe4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:30:48.572278 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:48.573225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8f575fec-934d-43b8-8c8a-f85943783147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:30:48.575487 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1371/2721] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:30:48 2022] DELETE /identity/v3/projects/d93668b12728448aa781f26f4d7bbe4e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:30:48.581281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:48.581877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:48.582022 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:48.613664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:48.614920 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:48.662888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:48.695425 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1351/2722] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:48.703362 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:48.703645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:48.704152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:48.728936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:48.730180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:48.739736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1372/2723] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:30:48.746124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:48.756892 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c38fd-5f8d-4f8c-9d4c-aa5106342a31 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:48.762791 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:48.763450 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:48.763930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:48.764229 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:48.765200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:30:48.765575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:48.767285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:48.781897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1352/2724] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:30:48.786112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:48.799667 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785722a-8a2d-484c-bda4-38490f72dc21 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:48.806842 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:48.809917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:48.809917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:48.809917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:48.810755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:48.813474 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:48.860822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1373/2725] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:30:48 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:48.873764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:48.898487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4d2ca-979c-4611-aeca-38d51d402ffa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:49.014989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:49.015600 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.015873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.016144 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.016774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:49.019605 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:49.049427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1353/2726] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:30:48 2022] POST /identity/v3/users => generated 282 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:30:49.050746 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:49.064459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efbeb7-1cda-4e92-bc22-8f3dd6a55977 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:49.070651 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:49.071281 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.071736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.072010 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.072833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:49.074483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:49.084992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1374/2727] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:49.089100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:49.102130 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e948c29-7314-4c00-b659-4ff005b9aa41 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:49.107970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:49.108601 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.108879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.109155 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.109842 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:49.111480 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26aa3-2ba1-4b75-af9f-2c9603d57092 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:49.125628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1354/2728] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:30:49.128433 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:49.142555 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50bea5-fcca-4b11-9e1e-551bb33a5e78 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:49.148584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:49.149376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.149936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.150231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] PATH_INFO: `/v3/projects/87860ee49e344e4d8526b2b9b3d628bf/users/a702c8fbcbaa46d2925a54ba65e14c8f/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.168380 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87860ee49e344e4d8526b2b9b3d628bf, user_id=a702c8fbcbaa46d2925a54ba65e14c8f, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:49.171864 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:49.187052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1375/2729] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:30:49 2022] PUT /identity/v3/projects/87860ee49e344e4d8526b2b9b3d628bf/users/a702c8fbcbaa46d2925a54ba65e14c8f/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:30:49.198595 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2e92-f269-4899-aaa5-f9b18de2d998 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.198888 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2e92-f269-4899-aaa5-f9b18de2d998 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.199346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2e92-f269-4899-aaa5-f9b18de2d998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.257549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b17c2e92-f269-4899-aaa5-f9b18de2d998 None None] MFA Rules not processed for user `a702c8fbcbaa46d2925a54ba65e14c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:30:49.258904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b17c2e92-f269-4899-aaa5-f9b18de2d998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:49.324539 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1355/2730] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:30:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:30:49.337501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:30:49.339779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:30:49.409018 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cfbf-cc07-410a-84f3-1bc82b40ba74 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:30:49.412853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:30:49.413470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:30:49.413785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:30:49.414063 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:30:49.423465 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:49.426541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:30:49.428396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:30:49.440498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1376/2731] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:30:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:30:59.321486 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:30:59.442012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:04.238665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:04.239207 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:04.239294 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:04.261964 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] MFA Rules not processed for user `baa2e8e668894ea4b82fea3a2bc1fe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:04.263184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:04.323819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1356/2732] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:14.316506 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=25 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:20.309785 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:20.312091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:20.401192 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb047-3990-4521-b3bd-b119b2d27ad6 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:20.405172 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:20.405857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:20.406108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:20.406383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:20.408156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:20.493502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:20.495177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a293e9d-0125-4316-8212-436bf6c7c277 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:20.508644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1377/2733] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:22.146109 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:22.147928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:22.280180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca737e6-a8e7-45c9-b3ff-398422ddd7f9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:22.285771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:22.288455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:22.288455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:22.288455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] PATH_INFO: `/v3/users/dc1b4429caad4aa1bd8cdefa94134286` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:22.288931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1b4429caad4aa1bd8cdefa94134286)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:22.290898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:22.364469 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1b4429caad4aa1bd8cdefa94134286'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:22.365525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1b4429caad4aa1bd8cdefa94134286'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:22.366362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1b4429caad4aa1bd8cdefa94134286'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:22.380673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1b4429caad4aa1bd8cdefa94134286'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:22.389178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1b4429caad4aa1bd8cdefa94134286'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:22.410927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1357/2734] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:22 2022] DELETE /identity/v3/users/dc1b4429caad4aa1bd8cdefa94134286 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:22.415369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:22.415854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:22.415957 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:22.437420 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:22.438673 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:22.512051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1378/2735] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:22.523554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:22.525035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:22.595211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd307b-6d6a-4987-8264-3e7531b6b84c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:22.600365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:22.603930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:22.603930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:22.603930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:22.616901 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:22.618495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:22.630541 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1358/2736] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:23.361549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:23.363132 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:23.379057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b79c02-ebd7-4d68-8aa7-93456dce0019 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:23.385450 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:23.386269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:23.386551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:23.386820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] PATH_INFO: `/v3/projects/cb3054dcaa3e49598f8354def66953a7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:23.400977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3054dcaa3e49598f8354def66953a7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:23.403723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:23.455751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:23.468981 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3054dcaa3e49598f8354def66953a7', 'request_id': 'req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:23.473846 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:23.482376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3054dcaa3e49598f8354def66953a7', 'request_id': 'req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:23.485007 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be543-e2e1-41ad-b4d1-58f6ce4a7c7e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:23.494269 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3054dcaa3e49598f8354def66953a7', 'request_id': 'req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:23.513145 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:23.517508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] The token cache is being invalidate because project cb3054dcaa3e49598f8354def66953a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:23.517912 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:23.518725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:23.521000 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1379/2737] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:23 2022] DELETE /identity/v3/projects/cb3054dcaa3e49598f8354def66953a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:23.561402 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:23.562560 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:23.562910 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:23.563254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:23.565317 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:23.578295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:23.580159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:23.595391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1359/2738] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:27.692103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:27.693545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:27.707113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f8e7-ca04-4dcd-a2ad-c219b6bec9d6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:27.710664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:27.711266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:27.711542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:27.711811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:27.713590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:27.782565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:27.783994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:27.794755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1380/2739] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:31.317676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:31.319775 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:31.336484 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b28010-9421-4cea-bf05-b3c957ed60cd service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:31.341408 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:31.342669 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.342945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.343215 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] PATH_INFO: `/v3/users/7790a2ef47644d62863aafc890d03249` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.344404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7790a2ef47644d62863aafc890d03249)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:31.346438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:31.394766 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7790a2ef47644d62863aafc890d03249'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.395871 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7790a2ef47644d62863aafc890d03249'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.396697 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7790a2ef47644d62863aafc890d03249'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.409492 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7790a2ef47644d62863aafc890d03249'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.416210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91c00f58-facd-4686-88b5-dcf4b347833e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7790a2ef47644d62863aafc890d03249'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.428301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1360/2740] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:31 2022] DELETE /identity/v3/users/7790a2ef47644d62863aafc890d03249 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:31.618933 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:31.633887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd33c-f819-4949-b6fc-70fdfbb29abe service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:31.689889 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:31.690530 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.690815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.691092 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] PATH_INFO: `/v3/projects/aa6929ebd282417cb6ba149f50281240` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.692210 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6929ebd282417cb6ba149f50281240)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:31.694156 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:31.742286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6929ebd282417cb6ba149f50281240', 'request_id': 'req-2210c642-c086-4561-b99f-8e74b94a8381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.751391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6929ebd282417cb6ba149f50281240', 'request_id': 'req-2210c642-c086-4561-b99f-8e74b94a8381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.753972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:31.758960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6929ebd282417cb6ba149f50281240', 'request_id': 'req-2210c642-c086-4561-b99f-8e74b94a8381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.772453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] The token cache is being invalidate because project aa6929ebd282417cb6ba149f50281240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:31.773140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.773683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2210c642-c086-4561-b99f-8e74b94a8381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:31.775298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1381/2741] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:31 2022] DELETE /identity/v3/projects/aa6929ebd282417cb6ba149f50281240 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:31.780197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.780908 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.781241 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.803587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:31.805270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:31.876904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1361/2742] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:31.884683 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.884992 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.885265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.906068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:31.907284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:31.915369 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1382/2743] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:31.920208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:31.929932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febc58d-98b5-4e3e-9073-1699b7e5993a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:31.934545 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:31.935167 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.936119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.936369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.937270 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:31:31.937567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:31.939204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:31.950801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1362/2744] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:31:31.954988 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:31.964113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25777f2d-ef75-4849-bdb5-e445e0705804 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:31.968972 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:31.969567 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:31.969870 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:31.970140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:31.970762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:31.972623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.000076 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1383/2745] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:31 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:32.004127 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:32.014068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eadf9a-2e84-4486-8ca6-53c94c061630 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:32.034642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:32.072631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:32.073236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.073511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.073818 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.074421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:32.076353 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.096749 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1363/2746] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:32 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:32.100979 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:32.111714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e59f-007a-4750-ab45-fe19b7286d82 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:32.116314 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:32.116907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.117185 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.117456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.118725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:32.120321 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.130922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1384/2747] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:32.134728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:32.145460 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5bce7-3caa-4352-9ce9-093ca4da2bb4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:32.150044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:32.150631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.150904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.151168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.151797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:32.153371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538a0a-52fa-450d-8bf0-d6c615741e98 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.164148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1364/2748] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:32.168244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:32.178986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd427b55-9cc9-4a2b-8a71-9429ca77b1d6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:32.184025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:32.184668 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.184950 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.185225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] PATH_INFO: `/v3/projects/04a6906275c6465a8a512da45932ce5c/users/b9a7cb4605b54cad9908ab0bd729d74b/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.200383 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a6906275c6465a8a512da45932ce5c, user_id=b9a7cb4605b54cad9908ab0bd729d74b, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:32.203055 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.215959 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1385/2749] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:32 2022] PUT /identity/v3/projects/04a6906275c6465a8a512da45932ce5c/users/b9a7cb4605b54cad9908ab0bd729d74b/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:32.225694 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a839-e000-4676-99a8-047fe3951bf9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.226053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a839-e000-4676-99a8-047fe3951bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.226329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a839-e000-4676-99a8-047fe3951bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.254515 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0539a839-e000-4676-99a8-047fe3951bf9 None None] MFA Rules not processed for user `b9a7cb4605b54cad9908ab0bd729d74b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:32.255667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0539a839-e000-4676-99a8-047fe3951bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:32.319682 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1365/2750] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:32.330963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:32.332662 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:32.417392 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5287c-4e6b-4a2f-bd26-09695f5ce3c5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:32.431377 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:32.432175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:32.432559 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:32.432932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:32.441839 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:32.443436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:32.453510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1386/2751] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:40.301071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:40.301571 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:40.301675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:40.353547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:40.354895 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:40.418953 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1366/2752] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 22:31:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:40.465961 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:40.468014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:40.539030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:40.564603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa504a-beb0-4aaa-ace9-e633d7a8a061 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:40.572596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:40.572596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:40.572596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:40.572596 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:40.581537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:40.582894 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c541af-1f0d-437c-95b1-a54ae06834b7 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:40.592421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1387/2753] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:31:41.920288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:41.925876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:41.996615 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65f24-0500-47b4-83a4-5051ad5ebb8f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:42.002529 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:42.003264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.003731 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.004212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] PATH_INFO: `/v3/users/b9a7cb4605b54cad9908ab0bd729d74b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:42.005590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a7cb4605b54cad9908ab0bd729d74b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:42.007825 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:42.073416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a7cb4605b54cad9908ab0bd729d74b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.074275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a7cb4605b54cad9908ab0bd729d74b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.075024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a7cb4605b54cad9908ab0bd729d74b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.098446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a7cb4605b54cad9908ab0bd729d74b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.111767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a7cb4605b54cad9908ab0bd729d74b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.143094 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1367/2754] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:41 2022] DELETE /identity/v3/users/b9a7cb4605b54cad9908ab0bd729d74b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:42.148203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.148709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.149059 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:42.189035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:42.190319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:42.297857 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1388/2755] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:42.304971 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:42.306527 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:42.379718 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cf137-0a6b-4497-9c1b-8410f43ba3cb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:42.386635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:42.387293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.387619 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.387924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:42.399430 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:42.401015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da8923-55cd-4288-94a7-c193ef60701c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:42.412827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1368/2756] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:42.727665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:42.729817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:42.745227 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc9d2-8e48-42a1-a1ee-9cc165aa05de None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:42.755111 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:42.756162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.756609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.757215 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] PATH_INFO: `/v3/projects/04a6906275c6465a8a512da45932ce5c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:42.758612 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a6906275c6465a8a512da45932ce5c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:42.761088 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:42.825095 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a6906275c6465a8a512da45932ce5c', 'request_id': 'req-f399af5f-99b1-4cfb-b296-257e7b72225a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.834780 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a6906275c6465a8a512da45932ce5c', 'request_id': 'req-f399af5f-99b1-4cfb-b296-257e7b72225a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.842717 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a6906275c6465a8a512da45932ce5c', 'request_id': 'req-f399af5f-99b1-4cfb-b296-257e7b72225a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.856921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] The token cache is being invalidate because project 04a6906275c6465a8a512da45932ce5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:42.858460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.859195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f399af5f-99b1-4cfb-b296-257e7b72225a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:42.861174 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1389/2757] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:42 2022] DELETE /identity/v3/projects/04a6906275c6465a8a512da45932ce5c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:42.865941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.866303 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.866759 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:42.891808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:42.893055 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:42.918907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:42.972139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1369/2758] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:42.981139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:42.981435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:42.981755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.004160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:43.005389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.014143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1390/2759] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:43.018701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.031335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367cc89-9723-4fb2-abcc-484fb6d2e650 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.037805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.039438 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.039782 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.040201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.041253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:31:43.041567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.043671 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.061796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1370/2760] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:31:43.062589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.072724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2807387-1654-4ef2-8583-d7ae6fab6c08 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.079497 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.080126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.080402 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.080671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.081299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.083237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.129705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1391/2761] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:43.130827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.144220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44f92-6402-42d4-a62b-bfb4d8895056 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.214179 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.215128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.215754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.216177 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.217015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.219056 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.246159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1371/2762] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:31:43.248506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.260390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da4748-a8ff-425c-a186-7a608e05de67 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.269467 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.270237 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.270517 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.270786 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.271576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.274341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.289271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1392/2763] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:43.291423 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.304137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6e4-0969-492e-8a2b-c2697a1fe95e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.314147 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.314930 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.315240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.315493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.316347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.318362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.318513 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.321157 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.330841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1372/2764] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:43.335218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.335777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf37d6-4d2a-4db8-8a5e-46df86c75729 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.346265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8e43d-3c4c-4385-92af-af2260da5b27 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.346494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.347160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.347437 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.347711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] PATH_INFO: `/v3/users/a702c8fbcbaa46d2925a54ba65e14c8f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.349016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a702c8fbcbaa46d2925a54ba65e14c8f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.351051 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.352042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.352921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.353035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.353279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] PATH_INFO: `/v3/projects/93847a0e2d13444fadbaa75bd136f4d0/users/a13f57e1bab34c2a8f71c29c060c8201/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.371427 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:43.387211 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93847a0e2d13444fadbaa75bd136f4d0, user_id=a13f57e1bab34c2a8f71c29c060c8201, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.390297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2505ef-b6d6-4f83-9d26-362611d0063d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.423332 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1373/2765] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:43 2022] PUT /identity/v3/projects/93847a0e2d13444fadbaa75bd136f4d0/users/a13f57e1bab34c2a8f71c29c060c8201/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:43.433273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a702c8fbcbaa46d2925a54ba65e14c8f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:43.433828 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.434117 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.434382 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.445029 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a702c8fbcbaa46d2925a54ba65e14c8f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:43.464575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a702c8fbcbaa46d2925a54ba65e14c8f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:43.480876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] MFA Rules not processed for user `a13f57e1bab34c2a8f71c29c060c8201`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:43.482685 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.504719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a702c8fbcbaa46d2925a54ba65e14c8f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:43.505399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a1ea4ed1-21ee-4baf-9c55-91555575472f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a702c8fbcbaa46d2925a54ba65e14c8f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:43.512305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1393/2766] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:43 2022] DELETE /identity/v3/users/a702c8fbcbaa46d2925a54ba65e14c8f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:43.517305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.517633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.517973 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.546265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:43.547426 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.569807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:43.588730 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1374/2767] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:43.593590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.595045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.637770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1394/2768] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:43.648625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.650196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:43.699095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86811d9-fc23-4f3f-ba9a-c9440de4423e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.707639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.708265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.708546 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.708814 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.709444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.711404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.723486 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59d0e2-4dd0-4bd9-bac5-e0d5f124b780 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.733871 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.734699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.735163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.735622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.747382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.749184 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.753805 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1375/2769] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:31:43.755112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.769930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1395/2770] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:43.860830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d1d5-f33c-4d52-8d55-72f1afeac0cd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.867549 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.868034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.868334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.868616 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.869251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.871230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.899767 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1376/2771] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:43 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:43.901739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.912837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab83848-0e87-4c78-9130-1052d5e486c4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.919462 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.920627 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.920906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.921176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.921850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.924030 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:43.939541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1396/2772] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:43.943399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:43.958470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228ad4-06c1-4464-a2ff-43f8d335e829 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:43.964534 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:43.965416 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:43.965837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:43.966137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] PATH_INFO: `/v3/projects/424a8c17c1fd4384b38a834a2baf7776/users/7a6a50516d2448c28d95914712a53820/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:43.984137 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=424a8c17c1fd4384b38a834a2baf7776, user_id=7a6a50516d2448c28d95914712a53820, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:43.988881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.011037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1377/2773] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:43 2022] PUT /identity/v3/projects/424a8c17c1fd4384b38a834a2baf7776/users/7a6a50516d2448c28d95914712a53820/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:44.014996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.029478 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b383bb-a26d-4ce0-a522-a273c5e85b9d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.067312 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:44.107852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.110447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.128407 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.129046 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.129504 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.129825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.130485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.133002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.165328 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1397/2774] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:44.167267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.184834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c06677-2b28-440a-8e85-30da955ace27 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.189531 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:44.193663 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.194797 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.195191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.195685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] PATH_INFO: `/v3/projects/424a8c17c1fd4384b38a834a2baf7776/users/7a6a50516d2448c28d95914712a53820/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.198066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=424a8c17c1fd4384b38a834a2baf7776, user_id=7a6a50516d2448c28d95914712a53820, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.201293 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3303a-c88b-48a0-af18-e724861c85d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.212456 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f96a-a6f5-4052-809f-97938e9e19ea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.215542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1398/2775] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:44 2022] PUT /identity/v3/projects/424a8c17c1fd4384b38a834a2baf7776/users/7a6a50516d2448c28d95914712a53820/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:44.226054 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.226931 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.228224 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.264002 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] MFA Rules not processed for user `7a6a50516d2448c28d95914712a53820`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:44.265876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.301173 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.302528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.303113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.303613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] PATH_INFO: `/v3/projects/87860ee49e344e4d8526b2b9b3d628bf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.305258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87860ee49e344e4d8526b2b9b3d628bf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.309103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.358685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1399/2776] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:44.372083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.375235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.392045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87860ee49e344e4d8526b2b9b3d628bf', 'request_id': 'req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:44.404627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87860ee49e344e4d8526b2b9b3d628bf', 'request_id': 'req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:44.420788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87860ee49e344e4d8526b2b9b3d628bf', 'request_id': 'req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:44.440052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] The token cache is being invalidate because project 87860ee49e344e4d8526b2b9b3d628bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:44.440400 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:44.441152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdacd2ae-081d-4d80-82ca-b70e7931ddd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:44.448163 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1378/2777] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:44 2022] DELETE /identity/v3/projects/87860ee49e344e4d8526b2b9b3d628bf => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:44.449346 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8f55-916c-4e9f-9a08-6fbb2ef103bd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.449741 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8f55-916c-4e9f-9a08-6fbb2ef103bd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.450021 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8f55-916c-4e9f-9a08-6fbb2ef103bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.471810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa974a-6a0a-416f-9c25-a5b87cdc3bb6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.472159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-443e8f55-916c-4e9f-9a08-6fbb2ef103bd None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:44.473500 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-443e8f55-916c-4e9f-9a08-6fbb2ef103bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.542600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.543626 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.544300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.544798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.546880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.552674 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:44.559931 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1379/2778] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:44.563929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.564297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.564652 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.594735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:44.596923 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:44.612849 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1380/2779] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:44.614068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.623379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.624831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.626693 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110248-a704-43e6-8de1-b0b86bb9c607 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.632236 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.633628 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.634135 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.634633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.635506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1400/2780] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:44.642214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:31:44.642621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.644476 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.659848 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1381/2781] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:31:44.664710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.683609 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff08927-10b0-4424-b173-e43842ca9365 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.695635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.696546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.696901 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.697287 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.698197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.700323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.748775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1401/2782] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:44 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:44.749858 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.766664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d82b0-7e58-4233-9405-4a7627052185 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.775338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:44.848701 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.849639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.849798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.852128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.852128 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.853567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.886794 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1382/2783] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:44 2022] POST /identity/v3/users => generated 295 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:31:44.890877 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.903914 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc179a6-20f3-443e-a864-55f144dfede0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.912897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.914154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.914625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.915276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.916277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.918521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.939850 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1402/2784] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:44.944080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:44.959569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a7ee-c843-4719-86b5-aa58c276670b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:44.965101 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:44.965873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:44.966165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:44.966474 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:44.967178 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:44.968946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37df05c-52f5-491a-ab80-ca9d964d85e6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:44.986703 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1383/2785] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:44.988273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:45.001160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451be18-595a-4a59-8e92-81dcc2850211 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:45.006922 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:45.007501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.007834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.008114 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] PATH_INFO: `/v3/projects/7a8dc1eecf1346b69f0166f5d72ded79/users/12a984ee12e14ef78fcfe487ebe901fd/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.045159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8dc1eecf1346b69f0166f5d72ded79, user_id=12a984ee12e14ef78fcfe487ebe901fd, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:45.056117 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:45.084025 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1403/2786] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:44 2022] PUT /identity/v3/projects/7a8dc1eecf1346b69f0166f5d72ded79/users/12a984ee12e14ef78fcfe487ebe901fd/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:45.095876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.096168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.096435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.135442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] MFA Rules not processed for user `12a984ee12e14ef78fcfe487ebe901fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:45.136780 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:45.141284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:45.143240 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:45.237396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1384/2787] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:45.249771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:45.252266 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:45.254040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c7df9-8f40-48a9-b874-01ad6c5ceede admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:45.261834 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:45.263067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.264069 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.264623 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] PATH_INFO: `/v3/users/a13f57e1bab34c2a8f71c29c060c8201` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.266844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13f57e1bab34c2a8f71c29c060c8201)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:45.269809 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:45.324337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:45.326634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13f57e1bab34c2a8f71c29c060c8201'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:45.335964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13f57e1bab34c2a8f71c29c060c8201'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:45.336161 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e378c27-564f-4308-964a-dff100d6f5f8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:45.341709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:45.342982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.343403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.343686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.351670 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13f57e1bab34c2a8f71c29c060c8201'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:45.358252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:45.359921 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:45.361209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13f57e1bab34c2a8f71c29c060c8201'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:45.362160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ac6770a-23ad-4742-90ff-167bf81b2f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13f57e1bab34c2a8f71c29c060c8201'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:45.369635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1404/2788] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:45 2022] DELETE /identity/v3/users/a13f57e1bab34c2a8f71c29c060c8201 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:45.370630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1385/2789] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:45.374390 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.378012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.378532 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.402851 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:45.404337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:45.484101 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1405/2790] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:45.494732 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:45.496914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:45.523937 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:45.586454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d3d21-8041-4388-a22b-b7207376d74a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:45.658310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:45.659247 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:45.659737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:45.660316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:45.672894 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:45.674389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:45.686354 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1386/2791] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:46.025519 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:46.027551 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:46.041711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98829cc-679e-47f5-a822-49d267d89ee7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:46.046963 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:46.047703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:46.048170 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:46.048516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] PATH_INFO: `/v3/projects/93847a0e2d13444fadbaa75bd136f4d0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:46.049808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93847a0e2d13444fadbaa75bd136f4d0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:46.052139 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:46.111660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93847a0e2d13444fadbaa75bd136f4d0', 'request_id': 'req-ba84a481-3528-448d-999c-5c18190b417e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.121771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93847a0e2d13444fadbaa75bd136f4d0', 'request_id': 'req-ba84a481-3528-448d-999c-5c18190b417e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.130564 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93847a0e2d13444fadbaa75bd136f4d0', 'request_id': 'req-ba84a481-3528-448d-999c-5c18190b417e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.146216 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] The token cache is being invalidate because project 93847a0e2d13444fadbaa75bd136f4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:46.146690 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.147415 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.149993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1406/2792] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:46 2022] DELETE /identity/v3/projects/93847a0e2d13444fadbaa75bd136f4d0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:46.154071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:46.164695 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef507bdb-b5aa-47d1-8289-f44316f2a5a3 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:46.229844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:46.230720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:46.230994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:46.231250 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] PATH_INFO: `/v3/users/7a6a50516d2448c28d95914712a53820` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:46.233169 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6a50516d2448c28d95914712a53820)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:46.235191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:46.293977 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6a50516d2448c28d95914712a53820'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.294873 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6a50516d2448c28d95914712a53820'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.295558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6a50516d2448c28d95914712a53820'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.309522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6a50516d2448c28d95914712a53820'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.318120 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f66d39e4-cab4-43f3-800e-9e04f797c3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6a50516d2448c28d95914712a53820'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.323049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:46.340670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1387/2793] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:46 2022] DELETE /identity/v3/users/7a6a50516d2448c28d95914712a53820 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:46.647599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:46.651016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:46.669253 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a481-3528-448d-999c-5c18190b417e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:46.738798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:46.755209 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:46.755866 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:46.756149 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:46.756421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] PATH_INFO: `/v3/projects/424a8c17c1fd4384b38a834a2baf7776` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:46.757577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=424a8c17c1fd4384b38a834a2baf7776)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:46.760067 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:46.814444 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '424a8c17c1fd4384b38a834a2baf7776', 'request_id': 'req-be54fd46-2db5-4dd6-a308-491c7f162297'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.844508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '424a8c17c1fd4384b38a834a2baf7776', 'request_id': 'req-be54fd46-2db5-4dd6-a308-491c7f162297'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.849633 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '424a8c17c1fd4384b38a834a2baf7776', 'request_id': 'req-be54fd46-2db5-4dd6-a308-491c7f162297'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.866971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] The token cache is being invalidate because project 424a8c17c1fd4384b38a834a2baf7776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:46.867509 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.868512 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be54fd46-2db5-4dd6-a308-491c7f162297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:46.872057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1407/2794] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:46 2022] DELETE /identity/v3/projects/424a8c17c1fd4384b38a834a2baf7776 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:46.879451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:46.879739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:46.880014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:46.910752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:46.915710 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:47.043339 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1388/2795] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:47.046020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.046419 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.046815 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.080599 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:47.081846 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:47.093180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1408/2796] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:47.101023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.114252 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e3f43-a352-4284-8f76-eda53db3ae99 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.126844 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.127796 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.128300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.128574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.129331 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:31:47.129664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.131444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.151513 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1389/2797] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:31:47.152616 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.164284 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead22de4-a745-4515-b2cc-1689f1870adc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.176831 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.177461 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.177778 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.178058 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.178770 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.180700 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.218052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1409/2798] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:47.220845 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.232095 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ead5b-9708-40c1-8de2-e25e29c831f9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.312924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.314124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.314124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.314124 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.314906 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.316728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.347968 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1390/2799] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:47.351985 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.362790 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed2785-4b3f-4de5-a03e-bf68606145cf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.367808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.368429 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.368710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.368986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.370121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.371271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.385547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1410/2800] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:47.386827 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.398550 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9768f7-a291-4fea-af87-df1767e3fb2f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.403675 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.404321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.404739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.405151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.405836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.407453 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a70abf-9f76-43c8-8ca5-71d7fdbde733 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.425369 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1391/2801] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:47.426650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.441804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae4ca-722b-45a6-bd2b-85f97f6ec3ad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.447528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.448199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.448483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.448813 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] PATH_INFO: `/v3/projects/ddc8418f81de4275bb1664d8003a6ebc/users/d5fd8f796e7246e4b6779a56fbb1bc35/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.469534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc8418f81de4275bb1664d8003a6ebc, user_id=d5fd8f796e7246e4b6779a56fbb1bc35, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.473394 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.487508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1411/2802] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:47 2022] PUT /identity/v3/projects/ddc8418f81de4275bb1664d8003a6ebc/users/d5fd8f796e7246e4b6779a56fbb1bc35/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:47.499917 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.500210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.500490 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.544195 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] MFA Rules not processed for user `d5fd8f796e7246e4b6779a56fbb1bc35`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:47.545700 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:47.612212 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:47.644343 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.645298 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1392/2803] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:47.739710 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa367c28-84ad-4fd0-89a1-6b2583aaa0ec admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.744682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.745376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.745766 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.746167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.747061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.749904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.775011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:47.787501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1412/2804] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:47.792783 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.807084 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac98d5a-e157-428e-989f-a1356e099877 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.878455 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.879315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.879734 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.880264 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.881078 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.883139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.910027 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1393/2805] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:47 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:47.915190 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.928289 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2ef7-d783-4dd1-8843-6c346fc4c522 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.934528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.935822 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.936339 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.936671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:47.937723 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:47.939382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:47.957654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1413/2806] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:47.959866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:47.976259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5cb75-3ddb-4330-be99-04a8fd4d5df5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:47.982991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:47.983907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:47.984201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:47.984612 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] PATH_INFO: `/v3/projects/c1b54029bd8442988875288c0d9baa21/users/37d61a97b91946919e6721a829863f03/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:48.008980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b54029bd8442988875288c0d9baa21, user_id=37d61a97b91946919e6721a829863f03, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:48.012463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:48.031112 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1394/2807] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:47 2022] PUT /identity/v3/projects/c1b54029bd8442988875288c0d9baa21/users/37d61a97b91946919e6721a829863f03/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:48.034907 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:48.046777 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e0df-222c-4a16-b416-353fdc515ded admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:48.118671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:48.119576 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:48.119989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:48.120354 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:48.121315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:48.124207 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e994d-f626-4f97-8a55-12c5974b5baf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:48.139016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1414/2808] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:31:48.143893 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:48.155754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d238e97-3c55-47c6-8434-a1ea7f62e9e9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:48.161945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:48.162756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:48.163010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:48.163367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] PATH_INFO: `/v3/projects/c1b54029bd8442988875288c0d9baa21/users/37d61a97b91946919e6721a829863f03/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:48.165843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b54029bd8442988875288c0d9baa21, user_id=37d61a97b91946919e6721a829863f03, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:48.168994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:48.194633 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1395/2809] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:48 2022] PUT /identity/v3/projects/c1b54029bd8442988875288c0d9baa21/users/37d61a97b91946919e6721a829863f03/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:48.196113 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:48.196615 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:48.197277 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:48.239719 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] MFA Rules not processed for user `37d61a97b91946919e6721a829863f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:48.240958 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:48.343393 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1415/2810] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:48.354371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:48.356248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:48.432107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeeaff-364c-4ac1-9298-59f63627fa4d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:48.436663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:48.437948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:48.438235 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:48.438498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:48.448371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:48.449885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832250-4151-42fa-a9ca-9f42f96616c5 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:48.460367 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1396/2811] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:49.089066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:49.167779 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92937a4-949a-45ad-8499-900ddfa52e0a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:49.172820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:49.173459 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:49.173804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:49.174083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] PATH_INFO: `/v3/users/d5fd8f796e7246e4b6779a56fbb1bc35` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:49.175245 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fd8f796e7246e4b6779a56fbb1bc35)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:49.177197 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:49.189323 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:49.243173 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fd8f796e7246e4b6779a56fbb1bc35'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.250887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fd8f796e7246e4b6779a56fbb1bc35'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.264483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fd8f796e7246e4b6779a56fbb1bc35'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.275820 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fd8f796e7246e4b6779a56fbb1bc35'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.276463 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fd8f796e7246e4b6779a56fbb1bc35'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.287494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:49.287813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:49.288060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:49.289678 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1416/2812] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:49 2022] DELETE /identity/v3/users/d5fd8f796e7246e4b6779a56fbb1bc35 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:49.308757 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:49.310114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:49.370447 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:49.391863 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1397/2813] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:49.402374 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:49.404643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:49.473936 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012f5e2-5853-4dbd-a72c-831b5ea236cb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:49.478016 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:49.478523 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:49.478802 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:49.479073 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:49.489665 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:49.491047 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:49.502217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1417/2814] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:31:49.851218 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:49.853029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:49.870834 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b187e19-a298-4b62-903d-803be2e409d9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:49.898242 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:49.898960 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:49.899295 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:49.899574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] PATH_INFO: `/v3/projects/ddc8418f81de4275bb1664d8003a6ebc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:49.903739 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc8418f81de4275bb1664d8003a6ebc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:49.905887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:49.966907 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc8418f81de4275bb1664d8003a6ebc', 'request_id': 'req-689d6c6b-223e-4a5f-8504-9214aeac5209'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.975180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc8418f81de4275bb1664d8003a6ebc', 'request_id': 'req-689d6c6b-223e-4a5f-8504-9214aeac5209'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:49.987639 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc8418f81de4275bb1664d8003a6ebc', 'request_id': 'req-689d6c6b-223e-4a5f-8504-9214aeac5209'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.000683 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] The token cache is being invalidate because project ddc8418f81de4275bb1664d8003a6ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:50.000981 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.001702 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.005966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1398/2815] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:49 2022] DELETE /identity/v3/projects/ddc8418f81de4275bb1664d8003a6ebc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:50.010542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:50.023376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ddd5-98d7-4d9a-acca-e7102ce932dd service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:50.101196 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:50.101881 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:50.102163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:50.102435 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] PATH_INFO: `/v3/users/37d61a97b91946919e6721a829863f03` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:50.104225 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d61a97b91946919e6721a829863f03)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:50.106282 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:50.148084 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:50.171211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d61a97b91946919e6721a829863f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.181611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d61a97b91946919e6721a829863f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.197727 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d61a97b91946919e6721a829863f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.207103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d61a97b91946919e6721a829863f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.207926 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8211bd0e-8713-418e-bdff-8de76a9dd0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d61a97b91946919e6721a829863f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.218248 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1418/2816] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:31:50 2022] DELETE /identity/v3/users/37d61a97b91946919e6721a829863f03 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:50.581561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:50.583754 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:50.628180 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6c6b-223e-4a5f-8504-9214aeac5209 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:50.700301 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:50.701096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:50.701280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:50.701552 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] PATH_INFO: `/v3/projects/c1b54029bd8442988875288c0d9baa21` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:50.702866 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b54029bd8442988875288c0d9baa21)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:50.705558 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:50.761387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b54029bd8442988875288c0d9baa21', 'request_id': 'req-2e78a629-40de-4676-8063-2ac68ef1795f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.769158 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b54029bd8442988875288c0d9baa21', 'request_id': 'req-2e78a629-40de-4676-8063-2ac68ef1795f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.778069 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b54029bd8442988875288c0d9baa21', 'request_id': 'req-2e78a629-40de-4676-8063-2ac68ef1795f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.789259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:50.793654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] The token cache is being invalidate because project c1b54029bd8442988875288c0d9baa21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:31:50.794887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.795896 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2e78a629-40de-4676-8063-2ac68ef1795f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:31:50.801387 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1399/2817] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:31:50 2022] DELETE /identity/v3/projects/c1b54029bd8442988875288c0d9baa21 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:50.803103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:50.803352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:50.803634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:50.834155 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:50.835389 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:50.911928 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1419/2818] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:50.919119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:50.919982 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:50.920273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:50.946497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:50.947764 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:50.955957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1400/2819] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:50.960235 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:50.971849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f850c-d9d5-4966-981b-108e2e1e3fde None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:50.976982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:50.977774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:50.978259 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:50.978541 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:50.979199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:31:50.979506 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:50.981189 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:50.994763 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1420/2820] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:31:50.999933 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.021790 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb6d1-5f9f-4ef3-9d02-48e249a5ca36 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.032276 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.034255 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.034572 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.034897 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.035554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.037832 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.081466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1401/2821] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:50 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:51.085918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.097994 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebd19-e46e-4fa7-b665-3fde97d54baa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.171917 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.172574 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.172794 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.173068 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.173837 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.177201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.205611 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1421/2822] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:51 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:31:51.210393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.224315 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6ee12-7d2d-4b3f-bfeb-7e7ca2200cb8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.230751 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.231485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.232344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.232344 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.233019 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.234833 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.249992 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1402/2823] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:51.251751 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.263020 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea8f57-3c29-40e9-8966-4366036ae24f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.271243 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.271887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.272175 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.272432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.273089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.275177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a1f17-2e71-4ab0-a1f5-a80b7113a1f5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.292301 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1422/2824] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:51.296302 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.311817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d190d4e-4f61-4006-95c8-4f6580af96ff admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.316281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.316948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.317225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.317499 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] PATH_INFO: `/v3/projects/f33d1d28d9eb4752ac0fdbf7f9ba5e7e/users/289b4880881345eaad2b25af12eb61e2/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.335246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33d1d28d9eb4752ac0fdbf7f9ba5e7e, user_id=289b4880881345eaad2b25af12eb61e2, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.338251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.357761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1403/2825] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:51 2022] PUT /identity/v3/projects/f33d1d28d9eb4752ac0fdbf7f9ba5e7e/users/289b4880881345eaad2b25af12eb61e2/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:51.363128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f37f5-187a-405e-9199-a4ab76fb5639 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.363584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f37f5-187a-405e-9199-a4ab76fb5639 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.363987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f37f5-187a-405e-9199-a4ab76fb5639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.393572 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1f8f37f5-187a-405e-9199-a4ab76fb5639 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:51.394993 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1f8f37f5-187a-405e-9199-a4ab76fb5639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:51.407752 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:51.496725 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1423/2826] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:51.507676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:51.509477 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:51.601692 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19815cf-b703-400c-8b31-d18e520f836c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:51.607836 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:51.608772 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:51.609204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:51.609590 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:51.620720 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:51.622240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:51.635442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1404/2827] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:52.485772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:52.486964 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:52.487594 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:52.527276 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] MFA Rules not processed for user `289b4880881345eaad2b25af12eb61e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:52.533905 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:52.649496 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1424/2828] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:31:52.655146 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:52.664914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664f2fa-2f0c-48c6-9a95-c41ddb52703f service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:52.676390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:52.681269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:52.681709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:52.682092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:52.682865 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:52.684927 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:52.733635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1405/2829] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:31:52 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:52.738079 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:52.752560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0168f-b9f5-457f-9ba9-1ead5bcc51ce None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:52.835879 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:52.836499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:52.836781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:52.837136 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:52.837808 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:52.840094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:52.873288 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1425/2830] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:31:52 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:31:52.876986 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:52.890445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8a89f-5544-4c24-ba9e-0302c6ce1f68 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:52.896543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:52.897152 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:52.897497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:52.897898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:52.898611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:52.902323 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:52.918961 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1406/2831] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:31:52.924590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:52.937049 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71530365-5985-4ce1-a36e-bdf5ef3a75bf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:52.942660 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:52.943331 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:52.943613 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:52.943890 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] PATH_INFO: `/v3/projects/3a4418b99369493d8633d078bcbafbe5/users/b87714c97fff4b788b1cc5f0054ef459/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:52.960887 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4418b99369493d8633d078bcbafbe5, user_id=b87714c97fff4b788b1cc5f0054ef459, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:52.964040 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:52.980977 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1426/2832] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:52 2022] PUT /identity/v3/projects/3a4418b99369493d8633d078bcbafbe5/users/b87714c97fff4b788b1cc5f0054ef459/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:31:52.986045 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:52.997036 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d0515-13be-456b-8723-9b7a96ca6dd3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:53.064148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:53.064776 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:53.065061 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:53.065333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:53.066088 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:53.067915 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6abc3-fe9e-4c87-9f96-d977669ef848 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:53.084817 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1407/2833] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 22:31:53.086307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:53.102768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80791f-09bc-4029-9802-0f689657ec79 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:53.110305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:53.111501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:53.111943 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:53.112375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] PATH_INFO: `/v3/projects/3a4418b99369493d8633d078bcbafbe5/users/b87714c97fff4b788b1cc5f0054ef459/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:53.115421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4418b99369493d8633d078bcbafbe5, user_id=b87714c97fff4b788b1cc5f0054ef459, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:53.119707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:53.126999 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:53.137334 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1427/2834] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:31:53 2022] PUT /identity/v3/projects/3a4418b99369493d8633d078bcbafbe5/users/b87714c97fff4b788b1cc5f0054ef459/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:31:54.010068 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:54.010602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:54.010722 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:54.019622 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:31:54.044263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] MFA Rules not processed for user `b87714c97fff4b788b1cc5f0054ef459`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:31:54.046089 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:54.135204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1408/2835] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:31:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:31:54.146535 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:54.148355 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:54.216121 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0b40-18a7-4c8d-b9dc-8900d27505d5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:54.220577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:54.221438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:54.221852 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:54.222477 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:54.290954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:54.292585 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1951e1-a638-46c1-9b27-fa64eb405f93 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:54.303298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1428/2836] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:31:55.762765 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:31:55.764403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:31:55.780465 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d953-213d-4da0-9752-132b3ed1f2e1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:31:55.785017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:31:55.785752 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:31:55.786165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:31:55.787044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:31:55.797912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:31:55.799485 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:31:55.810687 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1409/2837] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:31:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:00.056365 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:00.056805 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:00.056920 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:00.080249 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:00.081521 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:00.143966 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1429/2838] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 22:32:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:00.156108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:00.157585 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:00.231511 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a915e7d-f146-4d31-ac5c-eb6f93518a4a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:00.236603 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:00.237213 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:00.237521 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:00.237829 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:00.250010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:00.251611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:00.260181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1410/2839] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:32:02.333483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:02.335044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:02.397617 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:02.401975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a3215-b17c-400c-9ef7-9cc688230f67 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:02.406996 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:02.407643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:02.407927 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:02.408206 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] PATH_INFO: `/v3/users/baa2e8e668894ea4b82fea3a2bc1fe9e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:02.409356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa2e8e668894ea4b82fea3a2bc1fe9e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:02.411326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:02.457531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa2e8e668894ea4b82fea3a2bc1fe9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.464489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa2e8e668894ea4b82fea3a2bc1fe9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.476592 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa2e8e668894ea4b82fea3a2bc1fe9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.483485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa2e8e668894ea4b82fea3a2bc1fe9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.484251 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa2e8e668894ea4b82fea3a2bc1fe9e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.490254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1430/2840] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:02 2022] DELETE /identity/v3/users/baa2e8e668894ea4b82fea3a2bc1fe9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:02.669742 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:02.682053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9f377-aa72-4eda-b83a-6bffef4de876 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:02.741262 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:02.741925 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:02.742208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:02.742483 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] PATH_INFO: `/v3/projects/61268e79778c4a20b3b7beca375d5706` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:02.743635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61268e79778c4a20b3b7beca375d5706)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:02.745532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:02.752380 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:02.795807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61268e79778c4a20b3b7beca375d5706', 'request_id': 'req-32e045f3-0ea6-4259-bb14-cdf6b9955453'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.802860 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61268e79778c4a20b3b7beca375d5706', 'request_id': 'req-32e045f3-0ea6-4259-bb14-cdf6b9955453'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.812654 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61268e79778c4a20b3b7beca375d5706', 'request_id': 'req-32e045f3-0ea6-4259-bb14-cdf6b9955453'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.824184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] The token cache is being invalidate because project 61268e79778c4a20b3b7beca375d5706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:02.824501 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.825129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-32e045f3-0ea6-4259-bb14-cdf6b9955453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:02.826665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1411/2841] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:02 2022] DELETE /identity/v3/projects/61268e79778c4a20b3b7beca375d5706 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:05.281105 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:05.282548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:05.295643 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9199b-a514-44aa-8670-c0e0eecdd07a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:05.351464 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:05.352096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:05.352379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:05.352657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] PATH_INFO: `/v3/users/12a984ee12e14ef78fcfe487ebe901fd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:05.353806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a984ee12e14ef78fcfe487ebe901fd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:05.355709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:05.401586 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a984ee12e14ef78fcfe487ebe901fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.408460 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a984ee12e14ef78fcfe487ebe901fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.420162 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a984ee12e14ef78fcfe487ebe901fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.423637 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:05.428736 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a984ee12e14ef78fcfe487ebe901fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.429470 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a984ee12e14ef78fcfe487ebe901fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.438097 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1431/2842] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:05 2022] DELETE /identity/v3/users/12a984ee12e14ef78fcfe487ebe901fd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:05.442617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:05.443113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:05.443208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:05.462418 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:05.463733 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:05.523948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1412/2843] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:05.533769 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:05.535194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:05.594705 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef7f2-016e-44a5-8e93-1f6f84d26dad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:05.598546 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:05.599502 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:05.599781 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:05.600082 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:05.609913 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:05.611333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee57407-817d-4a75-8f41-9af2809973ef service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:05.621102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1432/2844] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:05.913617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:05.915098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:05.927265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022852-4fc0-4f01-ab22-a80c9f418146 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:05.932049 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:05.933660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:05.933660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:05.933660 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] PATH_INFO: `/v3/projects/7a8dc1eecf1346b69f0166f5d72ded79` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:05.934912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8dc1eecf1346b69f0166f5d72ded79)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:05.936938 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:05.986148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8dc1eecf1346b69f0166f5d72ded79', 'request_id': 'req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.991647 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8dc1eecf1346b69f0166f5d72ded79', 'request_id': 'req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:05.999464 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8dc1eecf1346b69f0166f5d72ded79', 'request_id': 'req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:06.010627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] The token cache is being invalidate because project 7a8dc1eecf1346b69f0166f5d72ded79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:06.010916 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:06.011543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bd4495fd-2b4a-4bcd-a8b1-e502b57314fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:06.013028 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1413/2845] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:05 2022] DELETE /identity/v3/projects/7a8dc1eecf1346b69f0166f5d72ded79 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:06.018229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.018514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.018792 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.038211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:06.039405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:06.099186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1433/2846] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:06.107269 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.108171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.108451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.127883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:06.129092 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:06.137034 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1414/2847] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:06.141376 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.151134 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb340-8d46-462c-8fd3-7c730e9eb5f6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.156285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.156885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.157167 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.157441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.158127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:32:06.158434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.160045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.172706 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1434/2848] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:32:06.177199 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.186723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8933d2-8213-4bba-990d-c552ad0ad3a7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.191642 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.192280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.192554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.192819 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.193445 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.195360 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.204877 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:06.223706 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1415/2849] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:32:06 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:06.227904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.238430 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406216b2-fcb4-42d8-82b8-803db00dde55 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.294375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.295658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.295873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.296144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.296773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.298718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.320775 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1435/2850] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:32:06 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:06.324884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.336122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba76d4c-45f8-493e-a3a0-44a4d31d9f06 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.340800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.341404 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.342231 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.342508 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.343148 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.344747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.354543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1416/2851] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:06.358935 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.368854 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44f5a-2b70-45ff-949f-a921f4dcb74e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.373699 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.374296 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.374579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.374873 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.375514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.377085 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d644-0eac-402d-8378-78bc848789aa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.387545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1436/2852] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:06.392156 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.402300 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdaabf-d295-484f-9557-35bda2e6683f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.407113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.407738 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.408010 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.408279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] PATH_INFO: `/v3/projects/5a5733a14ade43e48f6a039a5075e63c/users/4b4e033fbc0d438ca9a3f954d53c891e/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.422650 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5733a14ade43e48f6a039a5075e63c, user_id=4b4e033fbc0d438ca9a3f954d53c891e, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.425279 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.439253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1417/2853] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:06 2022] PUT /identity/v3/projects/5a5733a14ade43e48f6a039a5075e63c/users/4b4e033fbc0d438ca9a3f954d53c891e/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:06.444342 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf6770-b8fd-44ed-ac15-714bc9c25216 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.444628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf6770-b8fd-44ed-ac15-714bc9c25216 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.444930 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf6770-b8fd-44ed-ac15-714bc9c25216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.453540 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:06.464951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-babf6770-b8fd-44ed-ac15-714bc9c25216 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:06.466195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-babf6770-b8fd-44ed-ac15-714bc9c25216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:06.524554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1437/2854] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:06.535015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:06.536728 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:06.593620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6a51-cd5c-463e-a874-6b469c2d91a7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:06.597427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:06.598080 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:06.598357 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:06.598627 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:06.608305 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:06.609709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:06.619328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1418/2855] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:10.727640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:10.728098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:10.728318 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:10.760102 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] MFA Rules not processed for user `4b4e033fbc0d438ca9a3f954d53c891e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:10.761363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:10.851554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1438/2856] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:10.856066 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:10.867662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd14a5-903c-4d51-8e65-b4aa1567576f service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:10.872532 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:10.873275 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:10.873663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:10.874017 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:10.874665 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:10.876625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:10.910433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1419/2857] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:32:10 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:10.911622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:10.923432 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fbc22-8ec2-4dbe-aa4f-69578b79c7a2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:10.989108 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:10.989880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:10.990164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:10.990621 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:10.991065 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:10.992982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:11.016452 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1439/2858] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:32:10 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 22:32:11.021002 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:11.038651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aa824-a1bb-4692-a1aa-4f87ffdf2623 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:11.043567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:11.044129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:11.044410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:11.045200 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:11.045899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:11.047544 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:11.059919 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1420/2859] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:11.064337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:11.075241 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c5fc-1f12-4e34-9dda-d34deb3a0758 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:11.081825 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:11.082619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:11.082984 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:11.083265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] PATH_INFO: `/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d/users/c6f06c43181b4170939742624b73fe71/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:11.099610 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6818cc88ab394f6dae1effa2f9bc5e1d, user_id=c6f06c43181b4170939742624b73fe71, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:11.102579 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:11.114229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:11.119338 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1440/2860] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:11 2022] PUT /identity/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d/users/c6f06c43181b4170939742624b73fe71/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:11.123735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:11.134182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c8191-1f3e-4b35-b166-d42dafba7fa8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:11.194631 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:11.195248 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:11.195525 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:11.195798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:11.196495 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:11.198091 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:11.209210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1421/2861] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:11.213547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:11.225490 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e973fc-3060-4070-a7cc-420b8c1bb064 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:11.230634 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:11.231299 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:11.231578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:11.231853 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] PATH_INFO: `/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d/users/c6f06c43181b4170939742624b73fe71/roles/a8f69336710f474db0973fac38e02b80` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:11.233915 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6818cc88ab394f6dae1effa2f9bc5e1d, user_id=c6f06c43181b4170939742624b73fe71, role_id=a8f69336710f474db0973fac38e02b80)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:11.236740 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:11.248772 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1441/2862] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:11 2022] PUT /identity/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d/users/c6f06c43181b4170939742624b73fe71/roles/a8f69336710f474db0973fac38e02b80 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:13.220723 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:13.223747 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:13.246526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855259f5-7028-4a10-99f0-86f2a498c54d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:13.267493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:13.332761 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:13.333881 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:13.334686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:13.336184 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] PATH_INFO: `/v3/users/289b4880881345eaad2b25af12eb61e2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:13.337712 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] RBAC: Authorizing `identity:delete_user(user_id=289b4880881345eaad2b25af12eb61e2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:13.339840 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:13.409970 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289b4880881345eaad2b25af12eb61e2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.411461 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289b4880881345eaad2b25af12eb61e2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.412287 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289b4880881345eaad2b25af12eb61e2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.429774 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289b4880881345eaad2b25af12eb61e2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.438824 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289b4880881345eaad2b25af12eb61e2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.461680 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1422/2863] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:13 2022] DELETE /identity/v3/users/289b4880881345eaad2b25af12eb61e2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:13.653577 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:13.655451 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:13.676601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b56481-a355-490f-ac59-40048744e863 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:13.755194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:13.755849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:13.756131 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:13.756397 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] PATH_INFO: `/v3/projects/f33d1d28d9eb4752ac0fdbf7f9ba5e7e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:13.757646 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33d1d28d9eb4752ac0fdbf7f9ba5e7e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:13.760211 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:13.871191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33d1d28d9eb4752ac0fdbf7f9ba5e7e', 'request_id': 'req-bdc56c20-e550-4737-89e6-ecdc46a91fe7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.890971 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33d1d28d9eb4752ac0fdbf7f9ba5e7e', 'request_id': 'req-bdc56c20-e550-4737-89e6-ecdc46a91fe7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.898640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33d1d28d9eb4752ac0fdbf7f9ba5e7e', 'request_id': 'req-bdc56c20-e550-4737-89e6-ecdc46a91fe7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.912015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] The token cache is being invalidate because project f33d1d28d9eb4752ac0fdbf7f9ba5e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:13.912409 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.913098 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:13.918011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1442/2864] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:13 2022] DELETE /identity/v3/projects/f33d1d28d9eb4752ac0fdbf7f9ba5e7e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:13.919205 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:13.932760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b446-50d9-41e4-a779-0edb8e52677e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:14.040159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:14.041978 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:14.042260 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:14.043133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] PATH_INFO: `/v3/users/b87714c97fff4b788b1cc5f0054ef459` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:14.044394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87714c97fff4b788b1cc5f0054ef459)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:14.054208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:14.126373 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:14.133528 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87714c97fff4b788b1cc5f0054ef459'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:14.134561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87714c97fff4b788b1cc5f0054ef459'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:14.135237 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87714c97fff4b788b1cc5f0054ef459'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:14.165663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87714c97fff4b788b1cc5f0054ef459'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:14.180005 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77c9f10b-6961-412b-95b6-cf8c8c681e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87714c97fff4b788b1cc5f0054ef459'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:14.208946 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1423/2865] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:13 2022] DELETE /identity/v3/users/b87714c97fff4b788b1cc5f0054ef459 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:32:14.936817 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:14.940177 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:14.993730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56c20-e550-4737-89e6-ecdc46a91fe7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.141405 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.142127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.142412 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.143603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] PATH_INFO: `/v3/projects/3a4418b99369493d8633d078bcbafbe5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.145533 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4418b99369493d8633d078bcbafbe5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.147876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.183954 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:15.262707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4418b99369493d8633d078bcbafbe5', 'request_id': 'req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:15.290089 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4418b99369493d8633d078bcbafbe5', 'request_id': 'req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:15.301147 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4418b99369493d8633d078bcbafbe5', 'request_id': 'req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:15.316061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] The token cache is being invalidate because project 3a4418b99369493d8633d078bcbafbe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:15.316747 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:15.317876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4724f7d-9d41-4d31-b1e9-2822a5e24b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:15.319892 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1443/2866] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:14 2022] DELETE /identity/v3/projects/3a4418b99369493d8633d078bcbafbe5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:15.324561 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.325071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.325263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.363736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:15.367210 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:15.472924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1424/2867] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:15.480424 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.480709 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.480982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.507115 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:15.508346 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:15.518348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1444/2868] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:15.522564 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.535277 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9996-a64d-4a04-8621-1a4ec2e21f8f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.543626 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.545086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.545086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.545086 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.545726 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:32:15.546096 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.547797 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.568349 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1425/2869] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:32:15.569009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.580897 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ae364-2c06-4e06-b8de-ab3e31a98792 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.590730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.591363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.592261 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.592601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.593315 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.595273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.639015 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1445/2870] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:32:15 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:15.640787 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.662452 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675d959-cf65-4388-9a20-589711be31ae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.774259 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.775245 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.775822 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.776265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.777297 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.780966 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.818736 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1426/2871] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:32:15 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:15.823584 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.838630 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3090260-e6cb-444a-af19-709ea17e2148 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.845762 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.846469 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.846811 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.847285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.848009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.849904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.868035 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1446/2872] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:15.869719 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.885663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00f63b-27cb-46b4-ba2b-2d7f6595dd29 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.894602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.895249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.895729 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.896186 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.896855 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.898617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da549ea9-e60a-4cd4-a27d-ac1ddc9221e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.914905 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1427/2873] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:15.922771 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:15.935566 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad7f6-1485-4005-a67e-69870cfb3145 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:15.944479 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:15.945160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.945436 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.945869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] PATH_INFO: `/v3/projects/0dfb5913779d4ddd9ac7978bce09ce69/users/d26f2eaed0b8443396b78b0f8b7a6bd9/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:15.964583 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfb5913779d4ddd9ac7978bce09ce69, user_id=d26f2eaed0b8443396b78b0f8b7a6bd9, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:15.968199 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:15.986285 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1447/2874] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:15 2022] PUT /identity/v3/projects/0dfb5913779d4ddd9ac7978bce09ce69/users/d26f2eaed0b8443396b78b0f8b7a6bd9/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:15.998393 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147f61-11d0-4715-92b6-26e83522053b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:15.999071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147f61-11d0-4715-92b6-26e83522053b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:15.999405 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147f61-11d0-4715-92b6-26e83522053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.030071 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-36147f61-11d0-4715-92b6-26e83522053b None None] MFA Rules not processed for user `d26f2eaed0b8443396b78b0f8b7a6bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:16.031705 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-36147f61-11d0-4715-92b6-26e83522053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.045963 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:16.111093 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1428/2875] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:16.122268 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:16.124160 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.194560 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7a35-c7fc-4632-ab69-6b90147f422c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:16.199093 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:16.199885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.200233 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.200603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.212516 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:16.214239 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cdd66-1aaf-4dd1-8feb-6f4ed3e4903d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:16.225638 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1448/2876] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:16.248662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147f61-11d0-4715-92b6-26e83522053b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:16.316994 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147f61-11d0-4715-92b6-26e83522053b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:16.322291 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:16.322950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.323226 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.323497 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] PATH_INFO: `/v3/users/d26f2eaed0b8443396b78b0f8b7a6bd9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.325011 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26f2eaed0b8443396b78b0f8b7a6bd9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:16.328025 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:16.404962 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26f2eaed0b8443396b78b0f8b7a6bd9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:16.407928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26f2eaed0b8443396b78b0f8b7a6bd9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:16.409263 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26f2eaed0b8443396b78b0f8b7a6bd9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:16.432165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26f2eaed0b8443396b78b0f8b7a6bd9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:16.443505 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-950e91c4-7504-45b1-a614-4481709ae1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26f2eaed0b8443396b78b0f8b7a6bd9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:16.473203 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1429/2877] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:16 2022] DELETE /identity/v3/users/d26f2eaed0b8443396b78b0f8b7a6bd9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:16.477849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.479833 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.480265 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.510039 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:16.511810 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.595032 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:16.631344 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1449/2878] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:16.631575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.631575 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.632449 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.640305 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:16.641883 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.668427 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] MFA Rules not processed for user `c6f06c43181b4170939742624b73fe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:16.670440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.698370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:16.715304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81675c2-81a5-481b-8b14-a86e994c84e7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:16.722542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:16.723267 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.723671 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.724180 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.734970 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:16.736563 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:16.748495 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1450/2879] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 22:32:16.767516 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1430/2880] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:32:16.782967 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:16.785078 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:16.916382 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17680338-9849-4d66-8fa4-da22939b9dae service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:16.927408 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:16.928052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:16.928327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:16.928600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:16.930688 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:17.003324 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.005942 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300645f-4176-4f0d-8157-d75429ecaf19 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.018522 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1451/2881] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:17.072097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.074108 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:17.105763 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3038-90d9-4c6b-afdf-18b2c1f1c97b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.112103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.113625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.114116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.114466 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] PATH_INFO: `/v3/projects/0dfb5913779d4ddd9ac7978bce09ce69` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.116220 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dfb5913779d4ddd9ac7978bce09ce69)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.118468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.176258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dfb5913779d4ddd9ac7978bce09ce69', 'request_id': 'req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:17.183987 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dfb5913779d4ddd9ac7978bce09ce69', 'request_id': 'req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:17.194362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dfb5913779d4ddd9ac7978bce09ce69', 'request_id': 'req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:17.210547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] The token cache is being invalidate because project 0dfb5913779d4ddd9ac7978bce09ce69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:17.210945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:17.211792 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b5c9230-6d23-4fff-bdd8-0b16c38002c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:17.214341 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1431/2882] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:17 2022] DELETE /identity/v3/projects/0dfb5913779d4ddd9ac7978bce09ce69 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:17.218951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.219507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.219885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.247363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:17.250603 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:17.278178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:17.322126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1452/2883] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:17.330487 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.331014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.331014 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.351621 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:17.353097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:17.364389 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1432/2884] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:17.370001 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.381818 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac107bb6-c0c7-4da7-9788-f262d87abf5b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.387514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.388434 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.389298 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.389503 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.390181 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:32:17.390485 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.392191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.405642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1453/2885] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:32:17.409989 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.442334 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5d8f-9a68-47c8-8d2a-9373202786e1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.448204 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.449321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.449993 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.450321 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.451649 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.454201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.485932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1433/2886] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:32:17 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:17.490534 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.508916 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991982-0611-4226-b070-98dfc67a6e6a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.586531 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.587182 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.587508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.587795 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.588542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.590658 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.620191 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1454/2887] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:32:17 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:17.624189 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.634812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213aff-cd7b-4ca5-b69b-df5b293cd1c5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.641024 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.641801 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.642201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.642569 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.643379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.645258 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.658580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1434/2888] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:17.662685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.674358 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd4a1-57ff-4db3-b738-c3cd4a13694b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.682378 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.683186 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.683716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.683856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.684543 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.686600 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f0843-034d-478d-a872-c7c6629758cf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.702356 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1455/2889] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:17.706434 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.723463 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a773f-924a-4e43-bd6c-bd8276d9bbe9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:17.735890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:17.736735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.737082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.737446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] PATH_INFO: `/v3/projects/bb00011a7f424959b5202d09f3fe7411/users/ac33f6272b124e3ca1a2bbcb138e6fe3/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.755309 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb00011a7f424959b5202d09f3fe7411, user_id=ac33f6272b124e3ca1a2bbcb138e6fe3, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:17.758328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:17.787153 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1435/2890] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:17 2022] PUT /identity/v3/projects/bb00011a7f424959b5202d09f3fe7411/users/ac33f6272b124e3ca1a2bbcb138e6fe3/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 22:32:17.788141 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:17.788425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:17.788701 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:17.835129 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] MFA Rules not processed for user `ac33f6272b124e3ca1a2bbcb138e6fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:17.836484 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:17.959552 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1456/2891] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:32:17.974830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:17.977566 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:18.027778 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:18.048957 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a82a19-77b0-4637-892f-aaaa1a290931 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:18.059253 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:18.060922 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:18.063800 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:18.064198 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:18.080426 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:18.082115 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3a6a6-ce3a-4dd5-8cfc-da07b35842e6 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:18.099335 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1436/2892] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:20.835549 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:20.862087 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:20.901203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf0245-e4fb-45d4-856d-a74bcc00ebda None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:20.913640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:20.913640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:20.913640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:20.913640 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] PATH_INFO: `/v3/users/ac33f6272b124e3ca1a2bbcb138e6fe3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:20.914217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac33f6272b124e3ca1a2bbcb138e6fe3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:20.914217 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:20.964304 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac33f6272b124e3ca1a2bbcb138e6fe3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:20.971154 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac33f6272b124e3ca1a2bbcb138e6fe3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:20.983578 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac33f6272b124e3ca1a2bbcb138e6fe3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:20.991290 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac33f6272b124e3ca1a2bbcb138e6fe3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:20.991976 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac33f6272b124e3ca1a2bbcb138e6fe3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.002856 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1457/2893] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:20 2022] DELETE /identity/v3/users/ac33f6272b124e3ca1a2bbcb138e6fe3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:21.005015 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.005510 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.005640 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.027580 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:21.029384 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:21.092037 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1437/2894] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:21.101711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.103137 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:21.171363 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d80a7-06d2-4625-b5de-69ca7c4760f1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.175256 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.175876 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.176152 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.176421 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.187119 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.187919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e05a1-c392-4094-9ddf-73d0570f75a0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.197659 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1458/2895] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:21.480876 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.483219 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:21.499854 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d70b8-3942-452e-b28f-094b2c541c26 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.504514 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.505159 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.505440 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.505740 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] PATH_INFO: `/v3/projects/bb00011a7f424959b5202d09f3fe7411` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.506885 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb00011a7f424959b5202d09f3fe7411)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.508807 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.558543 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb00011a7f424959b5202d09f3fe7411', 'request_id': 'req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.564233 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb00011a7f424959b5202d09f3fe7411', 'request_id': 'req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.571533 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb00011a7f424959b5202d09f3fe7411', 'request_id': 'req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.583795 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] The token cache is being invalidate because project bb00011a7f424959b5202d09f3fe7411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:21.584273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.584914 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f27f9f59-70ca-4bd5-b02e-6a850e2c17a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:21.586435 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1438/2896] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:21 2022] DELETE /identity/v3/projects/bb00011a7f424959b5202d09f3fe7411 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:21.591379 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.592230 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.592508 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.612183 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:21.613425 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:21.667064 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:21.688735 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1459/2897] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:21.697891 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.698114 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.698399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.719031 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:21.720557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:21.728166 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1439/2898] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:21.732498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.742236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a99a-7d6a-4dd3-9ea3-e68e63770048 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.746744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.747337 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.748164 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.748455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.749096 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 22:32:21.749396 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.751024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.767902 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1460/2899] 10.222.0.82 () {58 vars in 1153 bytes} [Thu Aug 18 22:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 22:32:21.768756 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.785265 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dabe3-e453-4332-9d53-b859f7f7e6bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.790362 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.791040 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.791386 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.791788 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.792498 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.794567 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.825624 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1440/2900] 10.222.0.82 () {60 vars in 1153 bytes} [Thu Aug 18 22:32:21 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:21.829537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.839906 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35270-3b9e-4084-88d1-ab7a0fba9adf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.917083 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.917732 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.918012 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.918286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.918919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.920910 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.946514 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1461/2901] 10.222.0.82 () {60 vars in 1144 bytes} [Thu Aug 18 22:32:21 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 22:32:21.953658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:21.966551 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093a9da-1413-47f1-bc09-e4f2f998488b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:21.975370 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:21.977272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:21.977272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:21.977272 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:21.977862 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:21.979107 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:21.993709 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1441/2902] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:21.995755 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:22.007606 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a2f15-b5d6-4f9b-99f4-f39bad707f09 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:22.012686 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:22.014062 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:22.014352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:22.014625 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:22.015273 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:22.016872 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7f88d-34a5-4f1e-82f4-0b852cfc55c3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:22.028286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1462/2903] 10.222.0.82 () {58 vars in 1122 bytes} [Thu Aug 18 22:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 22:32:22.033191 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:22.043296 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dc8f2-0437-4118-88dd-644f8045c154 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:22.050251 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:22.051060 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:22.051230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:22.051503 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] PATH_INFO: `/v3/projects/817eab2135d748a3aaef16bf2b6a694a/users/9f5f64f1c3ed4f148a13da8db367da38/roles/4f258cbfb9214c3f8093f3eca5da61b9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:22.069390 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=817eab2135d748a3aaef16bf2b6a694a, user_id=9f5f64f1c3ed4f148a13da8db367da38, role_id=4f258cbfb9214c3f8093f3eca5da61b9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:22.072106 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:22.083383 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:22.099208 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1442/2904] 10.222.0.82 () {60 vars in 1483 bytes} [Thu Aug 18 22:32:22 2022] PUT /identity/v3/projects/817eab2135d748a3aaef16bf2b6a694a/users/9f5f64f1c3ed4f148a13da8db367da38/roles/4f258cbfb9214c3f8093f3eca5da61b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:22.101796 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c51e-96f7-4dea-8f74-d8a75f860334 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:22.102094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c51e-96f7-4dea-8f74-d8a75f860334 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:22.102375 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505c51e-96f7-4dea-8f74-d8a75f860334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:22.134505 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1505c51e-96f7-4dea-8f74-d8a75f860334 None None] MFA Rules not processed for user `9f5f64f1c3ed4f148a13da8db367da38`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:22.135726 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1505c51e-96f7-4dea-8f74-d8a75f860334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:22.199236 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1463/2905] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:22.210518 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:22.212225 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:22.268944 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111aee-f89e-45fb-89cc-a02cbe10c61d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:22.272771 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b6922a858f459cbbaf1d5cad9967d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:22.273379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:22.273686 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:22.273959 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:22.283201 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:22.284635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044a1a7-be34-4fd4-a28e-cce3e31df9c8 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:22.293410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1443/2906] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:23.848311 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f35f-4db8-4637-afc6-31e516fb7432 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:23.848859 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f35f-4db8-4637-afc6-31e516fb7432 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:23.849009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f35f-4db8-4637-afc6-31e516fb7432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:23.869066 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4375f35f-4db8-4637-afc6-31e516fb7432 None None] MFA Rules not processed for user `9f5f64f1c3ed4f148a13da8db367da38`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:23.870326 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4375f35f-4db8-4637-afc6-31e516fb7432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:23.882491 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1464/2907] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:32.289421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=26 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:33.645342 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:33.645965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:33.645965 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:33.665139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] MFA Rules not processed for user `4b4e033fbc0d438ca9a3f954d53c891e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:33.666493 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:33.722918 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1444/2908] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:33.863547 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:38.860204 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edc4f1-3aed-4ce5-98f1-73b80d9fe7fd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:38.860750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edc4f1-3aed-4ce5-98f1-73b80d9fe7fd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:38.860750 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edc4f1-3aed-4ce5-98f1-73b80d9fe7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:38.879595 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-41edc4f1-3aed-4ce5-98f1-73b80d9fe7fd None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:38.880806 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-41edc4f1-3aed-4ce5-98f1-73b80d9fe7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:38.934645 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1465/2909] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 22:32:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:38.956347 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:38.957808 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:39.029182 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fb131-6f9a-4ae3-8ace-9ad586678537 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:39.034470 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:39.035103 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:39.035379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:39.035645 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:39.044557 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:39.046292 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:39.054613 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1445/2910] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:32:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:32:39.573094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bdd19-0ae9-4fdc-8dd7-86ba0a874826 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:39.573403 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bdd19-0ae9-4fdc-8dd7-86ba0a874826 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:39.573714 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bdd19-0ae9-4fdc-8dd7-86ba0a874826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:39.594554 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-895bdd19-0ae9-4fdc-8dd7-86ba0a874826 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:39.595787 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-895bdd19-0ae9-4fdc-8dd7-86ba0a874826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:39.603110 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1466/2911] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 22:32:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 22:32:39.615887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:39.626899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8072b7-242f-4cae-9535-beac14286605 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:39.632044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:39.632636 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:39.633133 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:39.633406 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:39.641735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:39.643311 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:39.652206 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1446/2912] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:32:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:32:39.873195 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a359738-2de2-473b-9ebe-fc9763e630e5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:39.873702 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a359738-2de2-473b-9ebe-fc9763e630e5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:39.873798 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a359738-2de2-473b-9ebe-fc9763e630e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:39.893489 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6a359738-2de2-473b-9ebe-fc9763e630e5 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:39.894744 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6a359738-2de2-473b-9ebe-fc9763e630e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:39.902116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1467/2913] 10.222.0.82 () {58 vars in 1009 bytes} [Thu Aug 18 22:32:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:39.915651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:39.926913 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd879f5-5be0-4551-a63a-d25f2965ccc3 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:39.932123 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:39.932758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:39.933029 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:39.933293 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:39.942316 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:39.943884 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:39.952798 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1447/2914] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:32:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:32:40.280724 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:40.281011 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:40.281307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:40.301313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] MFA Rules not processed for user `1222623a682a4d85bd967e96b9277098`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:40.302313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:40.309455 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1468/2915] 10.222.0.82 () {58 vars in 967 bytes} [Thu Aug 18 22:32:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:40.322942 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:40.334268 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cc6c6-63ad-46ad-a490-d91d2b18594a service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:40.340333 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '156f961161644bc6a2f19e0b1a0a32c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:40.341394 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:40.341912 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:40.342437 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:40.353667 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:40.355254 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83908aed-3eae-4c33-9689-2087e60fa937 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:40.363841 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1448/2916] 10.222.0.82 () {60 vars in 1384 bytes} [Thu Aug 18 22:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 22:32:42.448145 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:42.509274 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b52f5-5adf-4b5d-abb4-fcbb8ff048b9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:42.514758 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:42.515404 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:42.515682 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:42.515960 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] PATH_INFO: `/v3/users/9f5f64f1c3ed4f148a13da8db367da38` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:42.517252 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5f64f1c3ed4f148a13da8db367da38)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:42.519168 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:42.567619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5f64f1c3ed4f148a13da8db367da38'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:42.574654 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5f64f1c3ed4f148a13da8db367da38'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:42.587279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5f64f1c3ed4f148a13da8db367da38'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:42.597844 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5f64f1c3ed4f148a13da8db367da38'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:42.598510 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5f64f1c3ed4f148a13da8db367da38'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:42.604804 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1469/2917] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:32:42 2022] DELETE /identity/v3/users/9f5f64f1c3ed4f148a13da8db367da38 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:42.609444 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:42.609898 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:42.610053 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:42.630151 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] MFA Rules not processed for user `29b6162ed5d14f26ae87d2e2d8c81d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:32:42.631403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:42.684113 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:42.703338 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1449/2918] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:32:42.715368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:42.717169 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:42.725143 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:32:42.781774 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e900b1-2bf9-4eaf-bf4d-034b62d58ecd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:42.785703 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:42.786327 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:42.786608 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:42.786882 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:42.796528 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:42.798044 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:42.807548 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1470/2919] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:42.970397 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:42.971941 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:42.984422 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496dc87-a7bf-443c-8c13-3aab8d93296b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:42.989100 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:42.990504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:42.990504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:42.990504 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] PATH_INFO: `/v3/projects/817eab2135d748a3aaef16bf2b6a694a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:42.991424 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] RBAC: Authorizing `identity:delete_project(project_id=817eab2135d748a3aaef16bf2b6a694a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:42.993526 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:43.048928 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817eab2135d748a3aaef16bf2b6a694a', 'request_id': 'req-48327c1d-3937-4e4d-8684-4baa71cbabff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:43.054473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817eab2135d748a3aaef16bf2b6a694a', 'request_id': 'req-48327c1d-3937-4e4d-8684-4baa71cbabff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:43.062048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817eab2135d748a3aaef16bf2b6a694a', 'request_id': 'req-48327c1d-3937-4e4d-8684-4baa71cbabff'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:43.078611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] The token cache is being invalidate because project 817eab2135d748a3aaef16bf2b6a694a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:32:43.078899 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:43.079536 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:32:43.081023 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1450/2920] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:32:42 2022] DELETE /identity/v3/projects/817eab2135d748a3aaef16bf2b6a694a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:32:51.061091 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:32:51.070116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e20343-578e-4010-aed3-436797a5ebf9 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:32:51.119244 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:32:51.119848 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:32:51.120126 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:32:51.120399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:32:51.122163 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:32:51.184947 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:32:51.186550 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:32:51.196264 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1471/2921] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:32:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:32:53.071273 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:33:01.191987 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=74 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:33:52.988280 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:33:52.990547 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:33:53.003072 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48327c1d-3937-4e4d-8684-4baa71cbabff admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:33:53.007142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:33:53.007843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:33:53.008187 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:33:53.008530 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:33:53.010322 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:33:53.020246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:33:53.021945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:33:53.030364 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1451/2922] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:33:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:33:53.143739 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:33:53.152730 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacab1cf-16d6-4c14-8a18-2a48e9510cfb service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:33:53.156650 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:33:53.157254 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:33:53.157536 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:33:53.157840 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:33:53.159622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:33:53.229869 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:33:53.231231 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:33:53.242737 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1472/2923] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:33:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:03.026454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:34:03.237368 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=34 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:34:29.896098 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:29.897522 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:30.024662 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934f0a7-9109-451e-9045-a87603837dc8 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:30.029698 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:30.030310 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:30.030583 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:30.030868 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:30.032573 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:30.044249 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:30.045843 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:30.055473 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1452/2924] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:40.051330 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=58 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:34:48.450494 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:48.459316 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f93e-9d90-447b-94c2-0409ae3d5c1c service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:48.463024 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:48.463622 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:48.463904 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:48.464178 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:48.465941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:48.479698 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:48.481116 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:48.492310 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1473/2925] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:49.675651 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:49.677119 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:49.688623 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0983-96c6-4e08-aee1-c04afbb4c673 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:49.692446 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:49.693044 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:49.693371 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:49.693673 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:49.695542 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:49.709281 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:49.710760 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9fd5f-23f4-4bbb-bc48-ccc09f463603 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:49.722661 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1453/2926] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:50.992862 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:50.995652 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:51.012636 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b62072-8e41-4479-8f59-bf527cde62c4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:51.019061 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:51.019685 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:51.020052 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:51.020406 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:51.025989 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:51.040333 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:51.042123 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:51.057483 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1474/2927] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:52.022122 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:52.022574 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:52.022679 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:52.042793 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] MFA Rules not processed for user `4b4e033fbc0d438ca9a3f954d53c891e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:34:52.043926 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:52.096991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1454/2928] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:34:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:34:52.406398 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:52.417655 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7d5b0-4151-4d94-8eed-c9a8fbbae77b service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:52.421279 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:52.421918 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:52.422203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:52.422481 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:52.424212 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:52.434336 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:52.435761 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:52.444166 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1475/2929] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:34:52.885052 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:34:52.898932 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17903769-99a1-4fdc-a1f2-9f348eacccd6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:34:52.904481 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:34:52.905097 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:34:52.905375 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:34:52.905674 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:34:52.907421 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:34:52.920365 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:34:52.921991 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:34:52.932329 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1455/2930] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:34:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:35:02.440539 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=18 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:02.927955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=37 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:05.541319 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:05.558447 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94cea3-a2b2-4406-ae46-3af55514dee6 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:05.564352 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:05.565218 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:05.565501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:05.565801 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:05.567601 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:05.580951 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:05.582575 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:05.594309 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1476/2931] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:35:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:35:15.589128 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:16.392136 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:16.406285 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491a522-4d6e-489f-a5ed-f25e34e26951 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:16.411980 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd441753da1af4f3c8f3b5fb4088d201b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:16.412587 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:16.412869 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:16.413142 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:16.414950 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:16.427904 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:16.429666 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:16.441554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1456/2932] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:35:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:35:20.336351 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:20.346565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baac81-0ac4-4f00-915d-0156d06a19f7 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:20.350307 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:20.350849 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:20.351127 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:20.351453 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:20.353144 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:20.417159 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:20.418641 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:20.428456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1477/2933] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:35:23.458399 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:23.460648 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:23.477042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec62a2-c5bd-4d27-90ae-732be1cdef76 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:23.483494 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:23.484391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:23.484813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:23.485372 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:23.487165 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:23.500150 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:23.501887 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:23.522042 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1457/2934] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:35:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:35:26.673716 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:26.675300 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:26.688768 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe5b07-5ec0-4846-85d4-6dcfae2d2807 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:26.694142 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:26.694824 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:26.695112 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:26.695391 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] PATH_INFO: `/v3/users/3e7526ef9eb0426a855f5b2b9be06a8d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:26.697140 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7526ef9eb0426a855f5b2b9be06a8d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:26.699103 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:26.745975 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7526ef9eb0426a855f5b2b9be06a8d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:26.752773 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7526ef9eb0426a855f5b2b9be06a8d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:26.764440 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7526ef9eb0426a855f5b2b9be06a8d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:26.771194 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7526ef9eb0426a855f5b2b9be06a8d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:26.771929 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7526ef9eb0426a855f5b2b9be06a8d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:26.778090 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1478/2935] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:35:26 2022] DELETE /identity/v3/users/3e7526ef9eb0426a855f5b2b9be06a8d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:35:26.953413 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:26.966139 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de5ab8-b073-4c72-b1f1-35fb6fb651e5 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:27.023653 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:27.024288 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:27.024594 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:27.024872 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] PATH_INFO: `/v3/projects/6c01cdc6c55b4fdf999fbc75c1f4b51b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:27.026035 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c01cdc6c55b4fdf999fbc75c1f4b51b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:27.027943 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:27.075852 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c01cdc6c55b4fdf999fbc75c1f4b51b', 'request_id': 'req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.081664 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c01cdc6c55b4fdf999fbc75c1f4b51b', 'request_id': 'req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.090620 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c01cdc6c55b4fdf999fbc75c1f4b51b', 'request_id': 'req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.102711 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] The token cache is being invalidate because project 6c01cdc6c55b4fdf999fbc75c1f4b51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:35:27.103538 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.104216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.105735 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1458/2936] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:35:26 2022] DELETE /identity/v3/projects/6c01cdc6c55b4fdf999fbc75c1f4b51b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:35:27.109707 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:27.111153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:27.124332 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b277e-e961-43ed-8186-c22e3bfaa7ad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:27.180438 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:27.180991 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:27.181271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:27.181545 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] PATH_INFO: `/v3/users/987001aac6a04044a176dbba2ba5703b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:27.182693 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=987001aac6a04044a176dbba2ba5703b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:27.184589 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:27.190229 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:27.235982 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987001aac6a04044a176dbba2ba5703b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.242791 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987001aac6a04044a176dbba2ba5703b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.255176 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987001aac6a04044a176dbba2ba5703b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.262733 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987001aac6a04044a176dbba2ba5703b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.263367 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987001aac6a04044a176dbba2ba5703b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.269202 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1479/2937] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:35:27 2022] DELETE /identity/v3/users/987001aac6a04044a176dbba2ba5703b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:35:27.441617 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:35:27.443129 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:35:27.456879 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de5b2f-3b4c-4b9d-987d-1c7193fe7b01 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:35:27.518468 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:35:27.519116 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:35:27.519396 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:35:27.519672 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] PATH_INFO: `/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:35:27.520812 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86572fbdf3b47f79be6d5c3c28371fd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:35:27.522768 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:35:27.529936 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:27.573758 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86572fbdf3b47f79be6d5c3c28371fd', 'request_id': 'req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.579635 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86572fbdf3b47f79be6d5c3c28371fd', 'request_id': 'req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.586882 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86572fbdf3b47f79be6d5c3c28371fd', 'request_id': 'req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.598658 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] The token cache is being invalidate because project e86572fbdf3b47f79be6d5c3c28371fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:35:27.598975 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.599602 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e9d24ebf-b1c7-47b1-ba23-a0deeecd9fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:35:27.601082 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1459/2938] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:35:27 2022] DELETE /identity/v3/projects/e86572fbdf3b47f79be6d5c3c28371fd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:35:37.265880 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:35:37.595637 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:36:32.621171 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:36:32.622664 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:36:32.698639 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d231b1-8d16-43f0-8396-989ced2800e1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:36:32.746482 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:36:32.747094 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:36:32.747372 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:36:32.747642 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:36:32.749386 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:36:32.759201 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:36:32.760628 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:36:32.768565 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1480/2939] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:36:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:36:42.764986 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=55 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:36:47.014230 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:36:47.014762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:36:47.014762 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:36:47.033328 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] MFA Rules not processed for user `4b4e033fbc0d438ca9a3f954d53c891e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 22:36:47.034611 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:36:47.090641 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1460/2940] 10.222.0.82 () {58 vars in 958 bytes} [Thu Aug 18 22:36:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 22:36:57.082433 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=25 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:38:31.040077 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:31.041501 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:31.052027 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732a4a-1111-4b69-bd44-184db210d210 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:31.055760 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:31.056359 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:31.056644 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:31.056919 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:31.058619 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:31.070045 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:31.071456 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:31.081445 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1481/2941] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:38:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:38:41.076492 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:38:44.397563 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:44.407663 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4243a-f03d-4cca-8208-a3050bc87daa None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:44.411454 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2837aa09ce6d4000985f475688f612e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:44.412067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:44.412359 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:44.412630 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:44.414391 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:44.478170 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:44.479596 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:44.489890 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1461/2942] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:38:47.519348 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:47.520921 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:47.598286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5bc49-c4a5-4aba-addf-5ec85744726d service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:47.603507 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:47.604165 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:47.604441 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:47.604711 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:47.606443 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:47.619568 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:47.621153 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:47.631692 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1482/2943] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:38:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:38:50.712830 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:50.714410 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:50.727314 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010149d-0183-4bdf-84ab-234ece9379a0 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:50.731924 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:50.732554 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:50.732831 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:50.733104 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] PATH_INFO: `/v3/users/4b4e033fbc0d438ca9a3f954d53c891e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:50.734246 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b4e033fbc0d438ca9a3f954d53c891e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:50.736168 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:50.783948 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b4e033fbc0d438ca9a3f954d53c891e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:50.784625 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b4e033fbc0d438ca9a3f954d53c891e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:50.785240 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b4e033fbc0d438ca9a3f954d53c891e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:50.797048 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b4e033fbc0d438ca9a3f954d53c891e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:50.803929 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b4e033fbc0d438ca9a3f954d53c891e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:50.817379 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1462/2944] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:38:50 2022] DELETE /identity/v3/users/4b4e033fbc0d438ca9a3f954d53c891e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:38:50.992203 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:51.005271 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1d7-b54c-42be-b662-fca1b22ff8d7 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:51.062590 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:51.063220 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:51.063499 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:51.063885 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] PATH_INFO: `/v3/projects/5a5733a14ade43e48f6a039a5075e63c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:51.065003 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5733a14ade43e48f6a039a5075e63c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:51.066941 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:51.113057 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5733a14ade43e48f6a039a5075e63c', 'request_id': 'req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.120308 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5733a14ade43e48f6a039a5075e63c', 'request_id': 'req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.126924 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5733a14ade43e48f6a039a5075e63c', 'request_id': 'req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.137718 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] The token cache is being invalidate because project 5a5733a14ade43e48f6a039a5075e63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:38:51.138009 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.138635 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.140266 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1483/2945] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:38:50 2022] DELETE /identity/v3/projects/5a5733a14ade43e48f6a039a5075e63c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:38:51.144414 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:51.145813 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:51.158883 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f72ae-6f23-4b65-808d-2599bba30253 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:51.215090 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:51.215713 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:51.215988 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:51.216256 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] PATH_INFO: `/v3/users/c6f06c43181b4170939742624b73fe71` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:51.217403 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f06c43181b4170939742624b73fe71)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:51.219451 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:51.225197 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:38:51.267537 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f06c43181b4170939742624b73fe71'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.268232 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f06c43181b4170939742624b73fe71'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.268837 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f06c43181b4170939742624b73fe71'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.281181 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f06c43181b4170939742624b73fe71'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.288026 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f06c43181b4170939742624b73fe71'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.301154 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1463/2946] 10.222.0.82 () {58 vars in 1224 bytes} [Thu Aug 18 22:38:51 2022] DELETE /identity/v3/users/c6f06c43181b4170939742624b73fe71 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:38:51.474014 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:38:51.475537 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:38:51.489498 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcce72-8e66-4a97-a83b-ebf39e1dbf05 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:38:51.510124 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:38:51.552341 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29b6162ed5d14f26ae87d2e2d8c81d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '940373c3ec634186892a510e45b0afa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:38:51.553019 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:38:51.553286 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:38:51.553542 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] PATH_INFO: `/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:38:51.554722 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6818cc88ab394f6dae1effa2f9bc5e1d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:38:51.556604 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:38:51.602657 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6818cc88ab394f6dae1effa2f9bc5e1d', 'request_id': 'req-ed54add4-2056-4480-95fa-998d1ba99d0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.611313 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6818cc88ab394f6dae1effa2f9bc5e1d', 'request_id': 'req-ed54add4-2056-4480-95fa-998d1ba99d0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.617026 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6818cc88ab394f6dae1effa2f9bc5e1d', 'request_id': 'req-ed54add4-2056-4480-95fa-998d1ba99d0f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.627937 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] The token cache is being invalidate because project 6818cc88ab394f6dae1effa2f9bc5e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 22:38:51.628228 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.628861 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed54add4-2056-4480-95fa-998d1ba99d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 22:38:51.630399 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1484/2947] 10.222.0.82 () {58 vars in 1233 bytes} [Thu Aug 18 22:38:51 2022] DELETE /identity/v3/projects/6818cc88ab394f6dae1effa2f9bc5e1d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 22:39:01.297676 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:39:01.624932 dv-cinder-853678-3-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 22:39:32.852737 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 22:39:32.854171 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:39:32.922638 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5147341-1549-48a2-8e17-eeafde21c3d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 22:39:32.979067 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbb44da1cc45218ce9bb43f11e6715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e30a6241ad1344af9a89bf9f50eb994e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 22:39:32.979670 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 22:39:32.979945 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 22:39:32.980214 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 22:39:32.982442 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 22:39:32.993366 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 22:39:32.994955 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d0391-74b5-4861-a014-274865560a8d service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 22:39:33.005222 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1464/2948] 10.222.0.82 () {60 vars in 1365 bytes} [Thu Aug 18 22:39:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 22:39:43.001216 dv-cinder-853678-3-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=58 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}